Astronomy & Astrophysics manuscript no.

aanda ©ESO 2024

October 24, 2024

[CII] line intensity mapping the epoch of reionization with the

Prime-Cam on FYST
II. CO foreground masking based on an external catalog
C. Karoumpis ⋆1 , B. Magnelli2 , E. Romano-Díaz1 , K. Garcia3 , A. Dev1 , J. Clarke4 , T.-M. Wang1 , T. Bădescu1 , D.
Riechers4 , and F. Bertoldi1

1
Argelander Institut für Astronomie, Universität Bonn, Auf dem Hügel 71, D-53121 Bonn, Germany
2
Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM, 91191, Gif-sur-Yvette, France
arXiv:2410.17330v1 [astro-ph.GA] 22 Oct 2024

3
Department of Astronomy, University of Florida, 211 Bryant Space Science Center, Gainesville, FL 32611, USA
4
I. Physikalisches Institut, Universität zu Köln, Zülpicher Straße 77, D-50937 Köln, Germany

October 24, 2024

ABSTRACT

Context. The Fred Young Submillimeter Telescope (FYST) line intensity mapping (LIM) survey will measure the power spectrum
(PS) of the singly ionized carbon 158 µm fine-structure line, [CII], to trace the appearance of the first galaxies that emerged during
and right after the epoch of reionization (EoR, 6 < z < 9).
Aims. We aim to quantify the contamination of the (post-)EoR [CII] LIM signal by foreground carbon monoxide (CO) line emission
(3 < Jup < 12) and assess the efficiency to retrieve this [CII] LIM signal by the targeted masking of bright CO emitters.
Methods. Using the IllustrisTNG300 simulation, we produced mock CO intensity tomographies based on empirical star formation
rate-to-CO luminosity relations. Combining these predictions with the [CII] PS predictions of the first paper of this series, we evaluated
a masking technique where the interlopers are identified and masked using an external catalog whose properties are equivalent to those
of a deep Euclid survey.
Results. Prior to masking, our [CII] PS forecast is an order of magnitude lower than the predicted CO contamination in the 225 GHz
([CII] emitted at z = 6.8 − 8.3) band of the FYST LIM survey, at the same level in its 280 GHz ([CII] emitted at z = 5.3 − 6.3) and
350 GHz ([CII] emitted at z = 4.1 − 4.8) bands, and an order of magnitude higher in its 410 GHz ([CII] emitted at z = 3.4 − 3.9) band.
For our fiducial model, the optimal masking depth is reached when less than 10% of the survey volume is masked at 350 and 410
GHz but around 40% at 280 GHz and 60 % at 225 GHz. At these masking depths we anticipate a detection of the [CII] PS at 350 and
410 GHz, a tentative detection at 280 GHz, whereas at 225 GHz the CO signal still dominates our model. In the last case, alternative
decontamination techniques will be needed.
Key words. Galaxies: evolution – Galaxies: statistics– Galaxies: star formation– Galaxies: high-redshift

1. Introduction of reionization, powering it through their ultraviolet (UV) emis-

sion. Despite this understanding, the intricacies of the interac-
In its first billion years, the Universe underwent numerous tion between early galaxy formation, evolution, and the reion-
changes. Following the Big Bang, the hot and ionized plasma ization process remain elusive. Key uncertainties include the
cooled down, leading to the formation of neutral hydrogen atoms energy emitted by the early stars, the fraction of this energy
(e.g., Peebles 1968; Zel’dovich et al. 1969; Seager et al. 2000). that escaped the early galaxies (e.g., Bouwens et al. 2016), as
This resulted in the Universe being in a state of darkness, known well as the distribution of those ionizing sources in space and
as the "Dark Ages." The advent of the first stars and galaxies, time. Unfortunately, these galaxies are too faint to be detected in
roughly 300 million years after the Big Bang, re-illuminated the large samples with current telescopes, including the state-of-the-
Universe (e.g., Harikane et al. 2022). What makes this period in art James Webb Space Telescope (JWST, Boylan-Kolchin et al.
the history of the Universe even more special is that the appear- 2015), which creates challenges for accurately estimating the
ance of the first light sources concurs with a phase transition in properties of their population and does not allow for tracing the
the intergalactic medium, from neutral to ionized (e.g., Becker large-scale structure in which they reside.
et al. 2001; Barkana & Loeb 2001; Becker et al. 2015; Fan et al. A more effective approach than traditional galaxy surveys
2006; Venemans et al. 2013), a period known as the epoch of for tracing both star formation in early galaxies and the large-
reionization (EoR). scale structure in which they reside is line intensity mapping
The connection between the reionization of the Universe and (LIM; see, e.g., Kovetz et al. 2019; Bernal & Kovetz 2022).
the emergence of the first light sources is well established in cur- This method measures the integrated emission of spectral lines
rent theoretical models (e.g., Zaroubi 2013). Early massive stars from galaxies within a given spatial-spectral resolution element
located in the first incipient galaxies were the primary drivers (voxel) using a spectro-imager. By focusing on the singly ionized
carbon 158 µm fine-structure line, [CII], which is the brightest
⋆
e-mail: [email protected] line in typical star-forming galaxies (e.g., Stacey et al. 1991) and
Article number, page 1 of 19
 A&A proofs: manuscript no. aanda

an indicator of star formation rate (SFR; e.g, De Looze et al. characteristics. To this end, we assumed that the FYST LIM sur-
2014; Le Fèvre et al. 2020; Béthermin et al. 2020; Schaerer vey will cover a region of the sky benefiting from an external
et al. 2020a; Bouwens et al. 2022), a three-dimensional (3D) to- catalog with similar stellar mass completeness limits as the Eu-
mography of galaxies during the early universe can be obtained clid deep fields (Euclid Collaboration et al. 2022). This external
(e.g., Karoumpis et al. 2022). Several LIM experiments, such as catalog provides the angular position, redshift, stellar mass and
the EXperiment for Cryogenic Large-Aperture Intensity Map- SFR of the galaxies and it is used to pinpoint and mask vox-
ping (EXCLAIM, Pullen et al. 2023), the Terahertz Intensity els with a high likelihood of containing bright CO emitters. By
Mapper (TIM, Vieira et al. 2020), the CarbON CII line in post- carefully simulating this technique, we not only assess its effec-
rEionization and ReionizaTiOn (CONCERTO) project (CON- tiveness in mitigating CO contamination, but also examine its
CERTO Collaboration et al. 2020), the Tomographic Ionized- influence on the recovered [CII] PS. This includes an analysis of
carbon Mapping Experiment (TIME, Sun et al. 2021), and the heightened measurement uncertainty resulting from diminished
Fred Young Submillimeter Telescope (FYST, CCAT-Prime Col- survey volume and the effects of a convolved PS attributable to
laboration et al. 2023), are underway to target [CII] emission the window function of the mask.
coming from EoR (6 < z < 9) and post-EoR (3 < z < 6 ) red- The paper is organized as follows. Section 2 introduces the
shifts. formalism adopted for the modeling of the [CII] and CO emis-
Many of these observatories will prioritize measuring the sion. Section 3 describes the construction of the mock external
variance in the Fourier mode of these 3D tomographies through catalog, and provides an overview of the masking technique.
their spherically averaged power spectrum (PS). This statistical Section 4 examines the statistical properties of the individual CO
metric, although relatively simple, offers powerful limits on the lines, the impact of masking on the CO PS, the targeted [CII]
underlying line luminosity function, given that the PS correlates PS, and the total line CO+[CII] PS. Finally, in Section 5, we
with both the first moment (for scales > 10 Mpc) and the second critically assess the limitations of our work and suggest poten-
moment (at scales < 10 Mpc) of the line luminosity function tial improvements and additions to our models. Throughout the
(Karoumpis et al. 2022). Such robust constrains of the luminos- paper, we adopt a ΛCDM cosmology with the same parameters
ity functions of [CII] will provide rigorous statistical constraints used in the TNG300 simulation (Pillepich et al. 2018): h = 0.71,
on the SFR of (post-)EoR galaxies—a pivotal, yet elusive pa- Ωb = 0.046, Ωm = 0.281, σ8 = 0.8, ΩΛ = 0.719, and ns = 0.963.
rameter in reionization models (e.g., CCAT-Prime Collaboration
et al. 2023; Karoumpis et al. 2022).
2. Generating the mock FYST LIM survey
In this context, where precise measurements of the [CII] PS
at different scales during and after the EoR will be possible, it The Fred Young Submillimeter Telescope (FYST, CCAT-Prime
is crucial to study how various cosmic galaxy evolution mod- Collaboration et al. 2023) is set to be a state-of-the-art, 6-meter
els influence this PS and can thus be tested by comparison with diameter telescope designed for submillimeter to millimeter ob-
future LIM measurements. To contribute to this effort, in the servations. Placed at 5600 meters on Cerro Chajnantor, FYST
first article of this series (Karoumpis et al. 2022), we generated will utilize a novel crossed-Dragone optical design (Dragone
a span of predictions by post-processing the dark matter halo 1978; Parshley et al. 2018) for fast and efficient wide-field-of-
catalog from the IllustrisTNG300 hydrodynamical simulation view sky mapping. One of its key instruments, the Prime-Cam
(TNG300; Pillepich et al. 2018). We then used our predictions receiver, will offer impressive spectroscopic and broadband mea-
to assess the feasibility of detecting the PS of the [CII] line from surement capabilities, enabling a mapping speed over ten times
galaxies at redshifts between 3 and 8 using the spectro-imager faster than existing facilities (Vavagiakis et al. 2018). Prime-Cam
of FYST. Our results, which are consistent with empirically mo- features seven instrument modules tailored to specific scientific
tivated predictions (Clarke et al. 2024), demonstrated promis- programs. Two of the modules, named Epoch of Reionization
ing potential for detecting the [CII] PS at the critical comoving Spectrometer (EoR-Spec, Huber et al. 2022), integrate a Fabry-
length scale of 10 Mpc in four selected FYST bands centered Perot interferometer (Perot & Fabry 1899; Zou et al. 2022) to
at redshifts 3.7, 4.3, 5.8, and tentatively at redshift 7 (i.e., 410, observe the [CII] line emission at z = 3.5 − 8.05 (210 to 420
350, 280, and 225 GHz). However, those predictions did not ac- GHz) with a resolving power of approximately ∆λ/λ = 100.
count for a significant challenge faced by the LIM technique: the EoR-Spec will target two 5 deg2 regions in a survey dedicated to
difficulty in distinguishing the contribution of (post-)EoR [CII] measuring the [CII] PS from (post-)EoR galaxies (CCAT-Prime
galaxies from the infrared (IR) continuum and carbon monoxide Collaboration et al. 2023). The chosen survey fields require sen-
(CO) rotational line emission (2 < Jup < 12) of galaxies lo- sitive auxiliary observations with extensive wavelength coverage
cated at the same solid angle but at lower redshifts (z = 0 − 5.7; to help remove foreground sources (see Sect. 3). Consequently,
see Fig. 1 and Table 1). While the IR continuum emission is the Extended-COSMOS (E-COSMOS, Aihara et al. 2018), and
not a major concern for the PS measurement as it is spectrally Extended-Chandra Deep Field South (E-CDFS, Lehmer et al.
smooth and its frequency-coherent spectrum can be fitted and 2005) contained in the Euclid Deep Field Fornax (EDF-F, Euclid
removed (Van Cuyck et al. 2023), the contribution of CO emit- Collaboration et al. 2022) are the selected fields for the FYST
ters presents a real challenge since the variance of their emission LIM survey. Existing deep coverage in various bands for these
is of the same order or even exceeds that of the [CII] galaxies fields will soon be augmented by forthcoming imaging and spec-
(e.g., Béthermin et al. 2022; Roy et al. 2023). troscopy data from present and future state-of-the-art observato-
In this paper, we utilized the halo catalog from our previ- ries like JWST, Roman, Euclid, and the Large Millimeter Tele-
ous work (Karoumpis et al. 2022), and by implementing empir- scope (LMT). For a comprehensive overview of both current and
ical relations between SFRs and CO line luminosities of galax- upcoming datasets, see CCAT-Prime Collaboration et al. (2023).
ies, we modeled the foreground line emission corresponding to In this context, realistic simulations of the FYST LIM survey
the same frequencies as the (post-)EoR [CII] emission. Draw- are crucial. They allow for the optimization of the survey strat-
ing from these predictions, we evaluated a foreground removal egy, the examination of its ability to detect the [CII] LIM signal
method where voxels containing luminous CO emitters are iden- during the (post-)EoR, and the testing of various foreground mit-
tified and masked using mock external catalogs with realistic igation strategies. To this end, we utilize the TNG300 simulation
Article number, page 2 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

to create a 4◦ × 4◦ cone of mock galaxies spanning from z ≈ 0 − 9 To quantify the expected sample variance, we employed an
(see Sect. 2.1). The [CII] line emission from these galaxies is analytical relation derived by Gkogkou et al. (2023) to directly
then incorporated, focusing on the z ≈ 3.4 − 8.3 range to which calculate the coefficient of variation of the PS (CVPS ) of the LIM
the FYST observed frequencies are sensitive (Karoumpis et al. tomography. In their study, Gkogkou et al. (2023) investigated
2022, hereafter Paper I; see Sect. 2.2 for a summary). Subse- the impact of field-to-field variance (which we refer to as sample
quently, we introduce the CO (Jup = 3 − 12) foregrounds which, variance) on CO and [CII] PS predictions for future LIM exper-
in the context of FYST, are emitted by galaxies in the z ≈ 0 − 5.7 iments. This was achieved by combining a 117 deg2 dark mat-
range (see Sect. 2.3). The constructed observable cone is then ter lightcone with a model of the infrared sky. They developed
transformed into mock FYST LIM tomographies (see Sect. 2.4), an analytical formula that estimates the expected CVPS of the
containing both the foreground CO and background [CII] emis- [CII] and CO LIM tomographies, given their dimensions. Upon
sion, that serves as the basis for evaluating our masking tech- inputting the survey parameters of a 40 GHz frequency cover-
nique. age and a 4◦ × 4◦ field into their formula, we find that the CVPS
for the clustering (shot noise) component of the CO PS peaks at
410 GHz, with a value of about 7% (4%), while for the [CII] PS,
2.1. The cone of mock galaxies the CVPS peaks at 225 GHz with a value of about 8% (7%).
Our predictions of line intensities are based on a single real-
In this study, we utilized the dark matter (DM) halo catalog pre- ization of this 4◦ ×4◦ field and therefore may be subject to sample
sented in Paper I, which contains all the essential results of the variance, due to the small number of galaxies traced. This is par-
TNG300 simulation needed to estimate the line luminosity of the ticularly true at low redshift (near z = 0) where the comoving
mock galaxies (i.e., angular position, redshift, M∗ , SFR) which volume covered by our cone is small, and at high redshift (near
are defined as the gravitationally bound substructures hosted z = 7) where star formation occurs predominantly in overdense
in the DM halos. This catalog corresponds to an observational regions. To quantify the expected sample variance, we employed
cone encompassing a sky area of 4◦ × 4◦ and a redshift range an analytical relation derived by Gkogkou et al. (2023) to di-
of z = 0 − 9, and it was constructed using the TNG300 halo rectly calculate the coefficient of variation of the PS (CVPS ) of
catalogs as building blocks and corrected for resolution effects the LIM tomography. In their study, Gkogkou et al. (2023) in-
with the TNG100 DM halo catalogs as a reference (Pillepich vestigated the impact of field-to-field variance (which we refer
et al. 2018). However, for the redshifts of the cone where galax- to as sample variance) on CO and [CII] PS predictions for future
ies emit the brightest CO lines in the frequency range of FYST LIM experiments. This was achieved by combining a 117 deg2
(i.e., z = 0 − 5.7), we recalculated the resolution corrections, dark matter lightcone with a model of the infrared sky. They de-
this time using the newly available TNG50 simulation (Pillepich veloped an analytical formula that estimates the expected CVPS
et al. 2019). The reason for this update was that the CO emission of the [CII] and CO LIM tomographies, given their dimensions.
originates from redshifts where the TNG100 simulation does not Upon inputting the survey parameters of a 40 GHz frequency
accurately reproduce the observed evolution of the cosmic SFR coverage and a 4◦ × 4◦ field into their formula, we find that the
density (Madau & Dickinson 2014), with the peak epoch ap- CVPS for the clustering (shot noise) component of the CO PS
pearing at earlier cosmic times in the simulation (z ≈ 3) than the peaks at 410 GHz, with a value of about 7% (4%), while for
observations (z ≈ 2). Owing to its 17 times better mass resolu- the [CII] PS, the CVPS peaks at 225 GHz with a value of about
tion, TNG50 closely follows the cosmic SFR density evolution 8% (7%). Nevertheless, in Sects. 2.2 and 2.3 we demonstrate
of Madau & Dickinson (2014). The same applies for our cone that uncertainties on the modeling of the CO and [CII] emission
after the new mass resolution corrections. of galaxies introduced much larger uncertainties on the CO and
[CII] PS. For example, in the cases of the CO PS at 350 GHz and
the [CII] PS at 225 GHz, the relative variability of the models in
Redshift vs Observed Frequency for [CII] and CO Lines
9 relation to their mean, (Pmax − Pmin )/(Pmax + Pmin ), is ≈ 85% and
8 [CII] Selected EoR-Spec bands ≈ 90% respectively. Therefore, while sample variance affects
LIM survey covering only 4◦ × 4◦ field, it does not significantly
7 affect the evaluation of the masking technique performed in this
paper.
6 12
Redshift (z)

5
11
10
4 9 2.2. The [CII] emission
8
3 7 The mock galaxy catalog detailed in Sect. 2.1 underwent vari-
6 ous post-processing methods to associate [CII] emission to its
2 5 galaxies. We restricted these calculations to the redshift range of
4
1 3 z ≈ 3.4 − 8.3, where the [CII] emission of galaxies is redshifted
into the observing bands of EoR-Spec. All the related methods
0
200 250 300 350 400 450 and calculations are described in detail in Paper I and here we
Observed Frequency (GHz) only summarize the most important steps. Firstly, SFRs were at-
tributed to mock galaxies in two ways. One approach used their
Fig. 1: Spectral lines observable within four selected EoR-Spec intrinsic TNG300 SFR, corrected to account for the mass reso-
bands (illustrated by red-shaded regions), and originated from lution limitation of TNG300 (see Sect. 2.1). The other approach
galaxies situated at different redshifts. The dashed numbered matched the mock galaxy abundance with the observed, dust-
lines represent transitions within the CO rotational ladder, with corrected ultraviolet luminosity function of high-redshift galax-
the corresponding numbers indicating the Jup for each specific ies (Bouwens et al. 2015), assigning luminosities to the mock
transition. These CO lines act as the foreground contaminant for galaxies which are subsequently converted into SFRs follow-
the [CII] (red thin dashed line) LIM survey. ing Kennicutt (1998). Secondly, the [CII] luminosities of the
Article number, page 3 of 19
 A&A proofs: manuscript no. aanda

mock galaxies were estimated from the SFRs using three differ- 2.3.1. The CO (1-0) luminosity
ent SFR-to-L[CII] relations: from a semi-analytic model of galaxy
formation (Lagache et al. 2018), from a hydrodynamical simu- We estimated the molecular gas content of a galaxy, relying on
lation of a high-redshift galaxy (Vallini et al. 2015), and from a the well-established relationship between molecular gas and star
high-redshift [CII] galaxy survey (Schaerer et al. 2020b). formation rate. Several studies have shown that the depletion
time (tdep = Mgas /SFR) does not depend significantly on the
The various galaxy-to-SFR and SFR-to-L[CII] relations trans- stellar mass of galaxies and only slightly increases from z ≈ 4
late into large variations of the amplitude of the expected [CII] to z ≈ 0 (e.g., Scoville et al. 2017; Tacconi et al. 2018; Kaasi-
PS (Paper I). Such variations render the assessment of the de- nen et al. 2019; Magnelli et al. 2020; Wang et al. 2022). There-
tectability of the [CII] PS more problematic but at the same time fore, changes in the SFR of galaxies with stellar mass and red-
demonstrate the utility of this signal in discriminating between shift are primarily driven by changes in their gas reservoir. This
all these models. In the following, our fiducial [CII] model refers allowed us to make the simplifying assumption that all galax-
to the combination of the recalibrated TNG300 SFR with the ies convert their molecular gas into stars at a constant universal
SFR-to-L[CII] relation from Vallini et al. (2015). It was chosen rate. To estimate the molecular gas masses of our mock galax-
as it produces results that closely align with the median value of ies, we multiplied their SFRs (the corrected intrinsic TNG300
our predictions (Paper I). SFRs presented in Sect. 2.1) by tdep = 570 Myr, corresponding
to the depletion time of z ≈ 2 galaxies with a stellar mass of
2.3. The CO emission M∗ ≈ 1010 M⊙ (Tacconi et al. 2018). We verified that even with
this simple approach, the cosmic molecular gas mass density in-
ferred from our cone of mock galaxies was compatible with the
observations of Riechers et al. (2019), Magnelli et al. (2020),
Cosmic Gas Mass Density Evolution Decarli et al. (2020), and Boogaard et al. (2023) (Fig. 2). This
Cosmic Gas Mass Density (M /Mpc3)

agreement is particularly true around the peak of the molecu-

lar gas mass density at z ≈ 2, from where the bulk of the CO
108
foregrounds originate. A more complex model that accounts for
changes in the depletion time with redshift and distance from the
main sequence (MS; for a definition, see Sect. 2.3.2) produces a
similar gas mass density evolution, but to avoid introducing ad-
ditional parameters, we adopted the simpler model of a constant
107 ASPECS-CO pilot depletion time.
ASPECS-CO LP The calculated molecular gas masses were then used to pre-
COLDz
Magnelli +20 dict the CO line emission of each mock galaxy. We adopted a
Boogaard +23 constant light-to-mass ratio between the specific luminosity of
This work the CO (1-0) line and the molecular gas mass,
106
0 1 2 3 4 Mgas
Redshift (z) αCO = ′ = 3.6 M⊙ /K km s−1 pc2 , (1)
LCO(1−0)
Fig. 2: Redshift evolution of the cosmic molecular gas mass den-
sity of galaxies. The blue horizontal line represents predictions as its value for the MS galaxies does not seem to change sig-
from our cone of mock galaxies, while observational constraints nificantly up to z ≈ 3 (Decarli et al. 2016), and is adopted by
are presented by rectangles. The gray rectangles are for the dust- the majority of the molecular gas measurements (Daddi et al.
based estimates of Magnelli et al. (2020), the light gray rect- 2015; Riechers et al. 2019; Decarli et al. 2020; Lenkić et al.
angles are for the COLDz CO survey (Riechers et al. 2019), 2020; Chung et al. 2022).
the light blue rectangle for the CO-based estimates of Boogaard There is observational evidence that starburst galaxies might
et al. (2023), the red-shaded rectangles are for the ASPECS CO require adjustments to their αCO and tdep . Specifically, a lower
pilot survey (Decarli et al. 2020), while the pink rectangles are tdep (≈ 150 Myr as suggested in Tacconi et al. 2018) and a re-
for the ASPECS CO LP survey (Decarli et al. 2020). The width duced αCO value of 0.8 (as suggested in Downes & Solomon
of the rectangles indicates the redshift bin size and the height the 1998). Interestingly, the ratio of the proposed depletion times
1-σ confidence region. (570/150) is roughly equivalent to the ratio of the αCO values
(3.6/0.8), meaning these changes may roughly offset each other.
Therefore, while incorporating these adjustments would not sig-
The CO molecule is the second most abundant molecule in nificantly alter the LCO(1−0) of our starbursts, it complicates the
the Universe, surpassed only by molecular hydrogen (H2 ). In the model further. For this reason, we have chosen to use the sim-
typical conditions of giant molecular clouds, the rotational tran- pler model of a constant depletion time and αCO . Nevertheless,
sitions of the CO that emit line emission are easily excited, mak- the effect of starburst galaxies is further examined in Sect. 2.3.2.
ing it the most widely used tracer of molecular gas in both the More generally, although we may underestimate the stochas-
local (e.g., Wang & Hwang 2020; den Brok et al. 2021) and high- ticity in our scaling relations for the CO(1-0) luminosities due to
redshift universe (e.g., Solomon & Vanden Bout 2005; Daddi the simple assumptions of constant tdep and αCO, the significant
et al. 2015; Aravena et al. 2016; Riechers et al. 2019). Its lu- uncertainties introduced in Sect. 2.3.2 make any additional scat-
minosity is proportional to its mass, assuming the number of ter in the CO(1-0) luminosities unnecessary for the scope of this
clouds is small enough for them not to overshadow each other study. There, by making some assumptions on the spectral line
and that they are close to virial equilibrium (e.g., Carilli & Walter energy distribution (SLED) of our mock galaxies, we use these
2013). The strong correlation between the CO line luminosities CO(1-0) line luminosities to predict the CO line luminosities in
of galaxies and their molecular gas content enables us to forecast transitions that pose the most significant foreground challenges
these foregrounds using our cone of mock galaxies. for our LIM survey (from Jup = 3 to Jup = 12).
Article number, page 4 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

2.3.2. The CO SLED tic star-forming galaxy with cold ISM conditions, slightly below
the MS. On the other hand, we used the CO SLED of NGC253, a
characteristic starburst galaxy with warm and dense ISM condi-
tion. This linear combination allowed us to create a continuum of
CO Spectral Line Energy Distribution
Specific Luminosity Ratio (L'CO(Jup) / L'CO(Jup = 1))

conditions that encompassed a broad range of galaxies from MS

NGC253
BzK (i.e., MS at z 2) to starburst. This approach is supported by observations, which
1.2 Milky Way report a correlation between the CO SLED shape of a galaxy and
1.0 its distance from the MS (e.g., Valentino et al. 2020; Cassata
et al. 2020).
0.8 The CO SLED of the MW and NGC253 used in our study
are displayed in Fig. 3. The MW observations up to Jup = 5 were
0.6 taken from Carilli & Walter (2013), and the
  SLED was extended
0.4 to Jup = 6 − 10 using the CO Jup − Jlow /CO(5 − 4) measure-
ment from Wilson et al. (2017). The CO SLED of NGC253 was
0.2 taken from Mashian et al. (2015). The difference between the
MW and NGC253 CO SLEDs, which is greater at Jup = 5, is due
0.0
to the contrasting star formation environments in these galaxies.
2 4 6 8 10 12
JUp The MW features a cold, more diffuse molecular gas compo-
nent, while the intense star formation of NGC253 creates a warm
Fig. 3: CO SLED of the Milky Way (black circles) and NGC253 and dense gas environment, leading to higher excitation of CO
(black squares). The SLEDs resulting from the linear combina- molecules and an enhancement mid- to high-J (Jup = 4 − 8)
tion of the two, following Eq. 3, are shown by solid lines, with transition emission.
µ varying from 0 to 1 in steps of 0.1. The red line represents the We assigned each mock galaxy a SLED resulting from its
combination (with µ = 0.4) that best fit the BzK galaxies (red distance from the MS and the linear combination of these two
points) and is used here as reference for high-redshift MS galax- extreme CO SLEDs. The mixing parameter (µ) was calibrated so
ies. that ∆MS = 0 yields a SLED consistent with typical MS galaxies
at z = 1.5 (BzK galaxies; Daddi et al. 2015), and ∆MS = 1.5
produces the SLED of NGC253. In turn, this yields,
The CO SLED of each galaxy is largely defined by the phys-
ical conditions (gas density and temperature) of its interstellar ∆MS + 1
medium (ISM; e.g., Narayanan & Krumholz 2014). It is estab- µ= . (2)
2.5
lished that due to the diverse range of conditions in the ISM, the
CO SLED of a galaxy often requires more than one tempera- The CO SLED resulting from the linear mixing is given by the
ture component to accurately describe it (Valentino et al. 2020). equation,
However, simulating the complex interactions of gas physics is ’galaxy
beyond the scope of this paper. log LCO(Jup −Jlow ) = (1 − µ) log LCO(J
’MW
up −Jlow )
Here we employed a model with a small number of free + µ log LCO(J
’NGC253
, (3)
parameters, yet sufficiently advanced to replicate the CO lumi- up −Jlow )

nosity function in various transitions observed by the Atacama ’galaxy ’MW ’NGC253
Large Millimeter/submillimeter Array (ALMA) Spectroscopic where LCO(Jup −Jlow ) , LCO(J up −Jlow )
, and LCO(Jup −Jlow )
are the CO(Jup −
Survey (ASPECS, Decarli et al. 2020; Boogaard et al. 2020; Jlow ) specific line luminosities of our mock galaxy, the MW, and
Riechers et al. 2020) at frequencies akin to our 225 GHz band NGC253, respectively. Figure 3 displays an example of the indi-
(see Fig. 4 and Table 1). To achieve this, we adopted an em- vidual CO SLEDs resulting from Eq. 3 (for µ = 0 − 1 with steps
pirical approach based on the galaxy position in the M∗ − SFR of δµ = 0.1). The linear combination highlighted in red corre-
diagram, which is a valuable tool for evaluating the properties of sponds to ∆MS = 0 (i.e. µ = 0.4), and aligns by design closely
a galaxy. The majority of the galaxies on this diagram follows a with the observed CO SLEDs from BzK galaxies.
so-called MS (Elbaz et al. 2007), wherein more massive galaxies This approach where each galaxy has its individual CO
tend to have higher SFRs. However, there are distinct exceptions: SLED constitutes our fiducial model. However, we also consid-
starburst galaxies, for instance, exhibit exceptionally high SFRs ered two extreme models in addition to our fiducial, creating a
for their stellar masses; or passive galaxies, that have low SFR larger range of possible values for the CO intensity tomogra-
for their stellar masses. Here, we defined our MSs by fitting a phies. In the "low-contamination" model, all galaxies have the
log-linear relation to the M∗ − SFR sequence followed by our SLED of a typical MS galaxy (i.e., µ = 0.4; BzK galaxies; Daddi
mock galaxies at different redshifts. Then, we parameterized the et al. 2015), while in the "high-contamination" model, all galax-
position of a galaxy in this diagram using their distance from the ies have the SLED of a typical starburst (i.e., µ = 1; NGC253).
MS, ∆MS = log(SFR/SFRMS ), where SFRMS is the MS SFR at To verify the accuracy of this approximation, we compared
the redshift and stellar mass of the galaxy. Galaxies close to and the CO luminosity function inferred from our fiducial, "low-"
below the MS have relatively cold ISM (e.g., Carilli & Walter and "high-contamination" models to that observed by ASPECS
2013; Magdis et al. 2021), while those above the MS, such as (Fig. 4). Our model predictions cover a good range of the 1-
starburst galaxies, have a warmer, denser ISM (e.g., Silich et al. σ interval compatible with the ASPECS measurements, with,
2009; Carilli & Walter 2013; Kamenetzky et al. 2017). Based however, a slight over-prediction of the faint end and under-
on this assumption, we modeled the CO SLEDs of our mock prediction of the bright end. This former discrepancy could be
galaxies by linearly combining the observed and well sampled due to the fact that ASPECS is a line-flux-limited CO survey,
CO SLEDs of a MS-like galaxy and a starburst-like galaxy. We which naturally becomes less sensitive and more incomplete at
used the CO SLED of the Milky Way (MW), as a characteris- the faint end of the CO luminosity function. The difference at
Article number, page 5 of 19
 A&A proofs: manuscript no. aanda

the bright end between our models and the survey has been re- external catalogs to identify the locations of the brightest fore-
ported previously in the semi-analytical approach of Popping ground sources and mitigates their influence on the targeted PS
et al. (2019); Béthermin et al. (2022); Gkogkou et al. (2023). by excluding the implicated voxels from further analysis.
As suggested by Davé et al. (2020), this could be due to the as- In our study, the masks were created based on a mock ex-
sumption of a constant αCO for all of our galaxies. In any case, ternal stellar mass-selected catalog (Sect. 3.1). In order to ex-
the galaxies of the brightest bin are few in number and therefore plore several depths of masking, for each CO line transition
constitute a small percentage of the overall number of interlop- in each FYST frequency band, we created two masks of dif-
ers that should be masked. This slight underestimation should ferent depth: the "bright" and the "complete" masking arrays.
not therefore significantly impact our results. To generate the "complete" masking arrays, we masked every
galaxy in our stellar mass-complete external catalog with red-
2.4. The [CII] and CO mock intensity tomographies shift zJup = (νJup /νobs ± 20 GHz) − 1, where Jup refers to the upper
level of the CO transition, and νobs is the central observed fre-
In order to translate the line luminosities of our mock galaxies quency of one of the four selected EoR-Spec bands.
into mock line intensity tomographies, we assumed that the in-
For the "bright" masking arrays, we employed the same
tensity of a line (either CO Jup = 3−12 or [CII]) in a given voxel,
method but limited it to galaxies above the MS, (i.e., ∆MS > 0).
positioned at a specific R.A., Dec, and ν, is expressed as:
This way to identify bright CO contaminants solely based on
 j
Lline
 ∆MS stemmed from the observation that the cumulative CO line
1  X 
Iline =   ∗ G(R.A., Dec, ν), (4) contamination from galaxies correlates with the MS boundary
∆ν0 (∆θb )2  j∈voxel 4π r2j (1 + z j )2 


(see e.g., Fig. 5). By exclusively masking galaxies above the
MS, we effectively limited our masks to those galaxies that sig-
where r j is the comoving distance of the j-th galaxy, which re- nificantly contaminate the targeted [CII] signal. This approach
j optimizes the survey volume available for our PS analysis. The
sides at a redshift z j and has a line luminosity Lline ; ∗ is the
symbol of the convolution; and G(R.A., Dec, ν) is the 3D Gaus- masking arrays, constructed using these two subsamples, are
sian function representing the angular and spectral resolution presented in Sect. 3.2, and the methodology behind this mask-
element of the EoR-Spec. To adhere to the Nyquist-Shannon ing strategy is discussed in further detail in Sect. 4.2.
sampling theorem (Shannon 1949), we selected the grid spac-
ing of the mock tomographies to be three times smaller than
the full width at half maximum (FWHM) of this 3D Gaus- 3.1. The mock external catalog
sian function. Since the summation of all galaxy luminosities
within a voxel essentially represents the integration of the in- The fields that will be targeted by the FYST LIM survey (CCAT-
tensity field over the volume of the voxel, we divide twice by Prime Collaboration et al. 2023) are the E-COSMOS (Aihara
∆θb = FWHMR.A. /3 = FWHMDec /3 = FWHMang /3 and once et al. 2018) and the E-CDFS (Lehmer et al. 2005), with the
by ∆ν0 = FWHMν /3 to account for the spatial and frequency di- latter potentially being extended to cover the whole EDF-F. To
mensions of the voxel. This ensures that the calculated intensity mimic the way these surveys would observe our cone of mock
is properly normalized by the volume of the voxel. galaxies, we simply applied to our mock galaxy catalog redshift-
We assumed that the FYST LIM survey will be divided into dependent stellar mass limits akin to those affecting an Euclid-
four tomographies for each of the four selected frequency bands deep-like survey (Euclid Collaboration et al. 2020). According
of the EoR-Spec. These bands are centered at 225 ± 20, 280 ± 20, to the latest forecasts, such a survey will have multi-wavelength
350 ± 20, and 410 ± 20 GHz. The average FWHM of the spectral photometric sensitivities that will be equivalent to those obtained
channels for these bands are 2.1, 2.7, 3.6, and 4.4 GHz, respec- today in the COSMOS field. By taking the stellar mass com-
tively. Additionally, the angular beam FWHM for these bands pleteness limits from the COSMOS2015 (Laigle et al. 2016)
are 0.88, 0.77, 0.65, and 0.62 arcmin, respectively. The intended and COSMOS2020 (Weaver et al. 2022) catalogs, we should
survey will cover an area of 5 deg2 . However, for our analysis, thus obtain realistic redshift-dependent stellar mass limits to ap-
we focused on a 16 deg2 field, as it offers a testbed with reduced ply to our mock galaxy catalog. These redshift-dependent limits
sample variance. It is important to note that when calculating are shown in Fig. 6, starting at ≈ 0.5 × 109 M⊙ at low redshifts
uncertainties (Sect. 4), we considered the sample variance and (z ≈ 0 − 2) and rising to ≈ 109 M⊙ at high-redshifts (z ≈ 2 − 4).
the white noise of the expected (i.e. two fields of 5 deg2 ) size In addition to the angular position, redshift information, and
and observational time (2000 hours for each field) of the FYST stellar mass, we also included in our mock external catalog the
LIM survey. This way we ensure that while the statistical param- SFR for each galaxy. Indeed, we anticipated that the Euclid Deep
eters of the foregrounds and the targeted signal approximate the Fields will provide SFR estimates for the majority of detected
global values, the impact of any low number statistics is effec- galaxies through the spectral energy distribution (SED) fit of all
tively captured within the uncertainty estimates. the available optical to near-infrared bands.
We note that the limited angular resolution of the EoR-Spec
3. The masking technique renders insignificant any potential offset between the CO posi-
tion and the optically-based position of galaxies in this external
While various alternative methods for addressing submillime- catalog, or any potential astrometry inaccuracies in the external
ter line foreground contamination have been proposed in the lit- catalog. Uncertainties on the angular position of our mock galax-
erature, including spectral template fitting (Kogut et al. 2015; ies were thus not included in our mock external catalog. More-
Cheng et al. 2020), blind masking (Breysse et al. 2015), and ma- over, given that the ∆z associated with our frequency channels
chine learning techniques (Moriwaki et al. 2020; Moriwaki & largely exceeds the anticipated photometric redshift accuracy of
Yoshida 2021; Zhou et al. 2023), we focus on what appears to be 0.002 × (1+z) from Euclid (Euclid Collaboration et al. 2021),
the most effective approach—targeted masking (e.g., Yue et al. uncertainties on the line-of-sight distance were also not included
2015; Silva et al. 2015; Sun et al. 2018). This technique employs in our mock external catalog.
Article number, page 6 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

Luminosity Function for CO(4-3), z 1 Luminosity Function for CO(5-4), z 2

2 ASPECS LP 2 ASPECS LP
This work This work
log [dex 1Mpc 3]

log [dex 1Mpc 3]

3 3

4 4

5 5

6 6

8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0
log (L'/[K km s 1 pc2]) log (L'/[K km s 1 pc2])

Luminosity Function for CO(6-5), z 2.5 Luminosity Function for CO(7-6), z 2.5
2 ASPECS LP ASPECS LP
This work 2
This work
log [dex 1Mpc 3]

log [dex 1Mpc 3]

3 3

4 4

5 5

6 6

8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0
log (L'/[K km s 1 pc2]) log (L'/[K km s 1 pc2])

Fig. 4: CO luminosity function for various transitions and redshifts. The blue line corresponds to our fiducial model, while the shaded
region indicates the range resulting from our optimistic and pessimistic models. Rectangles represent observational data from the
ASPECS LP survey (Decarli et al. 2020), with the width indicating the bin size and the height indicating the 1-σ confidence region.

Table 1: Rest-frame frequencies and redshift ranges of the brightest CO lines and [CII].

Line vr [GHz] ∆z(225 ± 20 GHz) ∆z(280 ± 20 GHz) ∆z(350 ± 20 GHz) ∆z(410 ± 20 GHz)
C II 1901.0 6.8 − 8.3 5.3 − 6.3 4.1 − 4.8 3.4 − 3.9
CO J =3−2 345.8 0.4 − 0.7 0.2 − 0.3 0.0 − 0.05 ···
CO J =4−3 461.0 0.9 − 1.2 0.5 − 0.8 0.2 − 0.4 0.1 − 0.2
CO J =5−4 576.3 1.4 − 1.8 0.9 − 1.2 0.6 − 0.7 0.3 − 0.5
CO J =6−5 691.5 1.8 − 2.4 1.3 − 1.7 0.9 − 1.1 0.6 − 0.8
CO J =7−6 806.7 2.3 − 2.9 1.7 − 2.1 1.2 − 1.4 0.9 − 1.1
CO J =8−7 921.8 2.8 − 3.5 2.1 − 2.5 1.5 − 1.8 1.1 − 1.4
CO J =9−8 1036.9 3.2 − 4.0 2.5 − 3.0 1.8 − 2.1 1.4 − 1.7
CO J = 10 − 9 1152.0 3.7 − 4.6 2.8 − 3.4 2.1 − 2.5 1.7 − 2.0
CO J = 11 − 10 1267.0 4.2 − 5.2 3.2 − 3.9 2.4 − 2.8 1.9 − 2.2
CO J = 12 − 11 1382.0 4.6 − 5.7 3.6 − 4.3 2.7 − 3.2 2.2 − 2.5
Notes. The rest-frame frequencies (νr ) of the brightest CO lines present in the selected EoR-Spec frequency bands and [CII], together with the
redshift range (∆z) from which their emission originate.

3.2. The masking array a voxel spilling over to neighboring voxels. Additionally, sharp
masks, like our binary masking arrays, produce high-frequency
components in their Fourier transform that are not well captured
The process of masking voxels containing bright CO contam- when sampling this space discretely. This causes high-frequency
inant is represented by a 3D array of zeros and ones, with details to alias into lower frequencies, distorting the Fourier-
zeros indicating voxels likely containing a bright CO emitter transformed data.
and thus requiring masking. This array is referred to as the
binary masking array, B. Here, through empirical testing, we A remedy for both issues is to smooth the binary mask using
found that the most effective voxel size for masking is that of a 3D Gaussian normalized such that its volume integral is equal
(FWHMν /3) × (FWHMang /3)2 . It is important to note that ob- to 1, a 3D equivalent of the 2D method used in CMB studies
served LIM tomographies undergo convolution with the beam (Ponthieu et al. 2011; Kim 2011). While Ponthieu et al. (2011)
of the telescope. This results in intensity from galaxies within recommends a Gaussian size double the map resolution, we have
Article number, page 7 of 19
 A&A proofs: manuscript no. aanda

this is expressed as:

Cumulative CO (6-5) Luminosity (L )

M = B × [1 − (1 − B) ∗ G(R.A., Dec, ν)], (5)
1010
Star Formation Rate (M /yr)

102
where 1 is an array of ones of identical dimensions to B. Through
109
this approach, the impact M has on the PS of a tomography
101
108 mirrors the impact B has on the PS of a perfectly deconvolved
MS = 0 map. This method bypasses deconvolution, a procedure that is
100 107 not only computationally intensive but that can also amplify the
noise of the observed data.
106
10 1

105 4. Results
10 2
104 4.1. Line intensities
1010 1011 1012
Stellar Mass (M ) z5[CII]
8 7 6 4
Fig. 5: SFR as a function of stellar mass with the bins color-
coded to reflect the cumulative CO (6-5) luminosity of the 2D 103 ALL
bin. The dashed line indicates the thresholds that define our B22
"bright" subsample with ∆MS > 0 (i.e., galaxies on and above [CII]

Mean Intensity [Jy/sr]

the MS) 4,5,6,7
102 3,8,9
10
Stellar mass completeness 11
1010
101
12
COSMOS2015
COSMOS2020
Stellar Mass (M )

This work 100

200 250 300 350 400 450

109
Frequency [GHz]
Fig. 7: Mean [CII] and CO line intensities as a function of
the observed frequency (or emitted redshift for [CII]). The red
dashed line depicts the [CII] fiducial model from Karoumpis
et al. (2022). Continuous gray lines represent the prediction for
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
Redshift(z) the CO emission, with line shade decreasing as Jup (gray num-
bers) increases. The thicker black line indicates the mean total
Fig. 6: Comparison of the stellar mass completeness limits ap- CO emission, while the Béthermin et al. (2022) prediction for
plied to our mock external catalog (red line) with those of the total CO mean intensity (green line) is included for comparison.
COSMOS2015 (light gray line Laigle et al. 2016) and COS-
MOS2020 (dark gray line Weaver et al. 2022) catalogs. The stel- To inform our masking strategy, we begin by analyzing the
lar mass completeness limits of the deep Euclid catalogs is ex- predicted intensities of the CO lines at the frequencies of the
pected to lie between these two lines. selected EoR-Spec bands (225±20 GHz, 280±20 GHz, 350±20
GHz, and 410 ± 20 GHz), that is, comparing the mean values
(Fig. 7) and CV (i.e., σI /µI ; Fig. 8) of their intensities to those
opted for a size consistent with the 3D beam of the EoR-Spec. In- of the targeted line, [CII].
deed, due to our broader binning in Fourier space, we do not face Variations in the mean intensities of spectral lines at specific
the same level of aliasing as CMB studies. This choice aligns observed frequencies are primarily due to two factors: the dif-
with the inherent resolution of the instrument and addresses po- ferent line luminosities of the galaxy populations emitting each
tential aliasing without significantly reducing the survey volume. line, and the different distances to each of these populations. In
However, as highlighted by Kim (2011), Gaussian smoothing Fig. 7, we see a clear trend in the evolution of line mean intensity
can risk foreground contamination, that is, pixels initially zero with observed frequency. As we shift towards lines with higher
in the binary mask might become non-zero after smoothing. To rest-frame frequency which therefore originate from higher red-
counteract this, we multiplied the smoothed mask with the orig- shifts, the gradient of their mean intensity as a function of fre-
inal binary mask. quency becomes markedly steeper. This steepening is a conse-
With the above considerations, we generated our mask as quence of more substantial changes between the luminosity dis-
follows: we subtracted B from a unit array, then convolved tances of galaxies of subsequent frequency channels when deal-
the outcome with the normalized 3D beam of the EoR-Spec, ing with higher redshifts.
G(R.A., Dec, ν). By subtracting the convolution result from an- Notably the mean intensities of CO(3-2) and CO(4-3) exhibit
other unit array and multiplying it by the original binary array, significant fluctuations at frequencies above 300 GHz. This vari-
we obtained the "applied masking array," M. Mathematically, ability can be attributed to their origin from redshifts z < 0.4,
Article number, page 8 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

where the volume encompassed by the observational cone is rel-

8 7 6
z5[CII] 4
atively small. Consequently, there is a limited number of emit-
ting galaxies within this volume, leading to intensity fluctuations 4
that are strongly driven by small number statistics. Furthermore,

Coefficient of Variation
CO(3-2) cannot be observed at frequencies above 345.8 GHz, its 101
rest frequency.
While the mean of the total CO intensity remains relatively
constant at all frequencies (≈ 103 Jy/sr) and is always domi-
nated by the low-J CO transition from low redshift galaxies, the
[CII] intensity shows a notable increase from about ≈ 10 Jy/sr at 5,6
225 GHz to ≈ 2000 Jy/sr at 410 GHz. We estimate that [CII] be- [CII]
comes the dominant line at > 364 GHz, a prediction that matches 7,8,12
9
well with the results of Béthermin et al. (2022) which quotes a 10,11
crossover at 358 GHz.
The variability in intensity distribution, as indicated by the
CV, is also critical for the efficiency of our masking method. A 200 250 300 350 400 450
high CV implies that the line intensity is concentrated in fewer Frequency [GHz]
voxels, while a lower CV indicates a more uniform voxel-to-
Fig. 8: Coefficient of variation of the [CII] (red) and CO lines
voxel distribution. This aspect becomes increasingly relevant as
(gray) intensities for the four FYST frequency bands. The gray
[CII] signal recovery becomes more challenging at lower fre-
numbers of decreasing opacity denote the Jup of the CO transi-
quencies (i.e. higher redshifts).
tion. The secondary x-axis indicating redshift applies solely to
The differences in CV arise from variations in the size of the
[CII] emitters.
galaxy populations contributing to the intensity of each line. This
size is contingent on the survey volume, which expands with in-
creasing redshift, and the number density of star-forming galax-
utilizes the spherically averaged PS, computed within the co-
ies, which peaks around z ≈ 3 in the case of our mock observa-
moving volume of the [CII] reference frame, thus standardizing
tional cone (in agreement with observations like Conselice et al.
the analysis frame to avoid constant conversions between dif-
2016). Consequently, the CV typically decreases from z ≈ 0
ferent lines. Contrasting with the approach in Paper I, here we
to z ≈ 4 and then increases for higher redshifts, as shown in
compute the PS for tomographies convolved with the 3D beam
Fig. 8 in the case of the [CII] line. For the bright CO lines (i.e.,
of the EoR-Spec, rather than incorporating the beam attenua-
Jup = 4 − 9, see Fig. 7), which are emitted by galaxies at z < 4,
tion into the uncertainty of the PS. This adjustment allows us to
only the decreasing trend appears. On the other hand, for the CO
simulate the effect that the interplay between the transfer func-
lines with Jup > 10, CV is constant with redshift at > 280 GHz
tions of the beam and the mask has on the measured PS. We
since they originate from z ≈ 2 − 3, where the survey volume
are focusing on the PS at k = 0.02 − 0.32 Mpc−1 , a scale that
expands slowly. Additionally, for these Jup , an increasing trend
is less affected by sample variance and masking bias. However,
appears at lower frequencies (< 280 GHz) where these lines are
our findings hold true at higher k-bins (k = 0.32 − 0.62 Mpc−1
emitted by galaxies at z > 4 where the number density of galax-
and k = 0.62 − 0.92 Mpc−1 ), suggesting that the effectiveness of
ies decreases rapidly.
the masking strategy is relatively constant across a broad range
Just by comparing the mean and the CV of the line inten-
of scales.
sities and before even applying any mask, we observed a clear
dichotomy. On the one hand, for the recovery of the [CII] emis- Mitigating CO contamination while preserving maximal sur-
sion originating from z < 5 galaxies, the situation is encouraging vey volume is far from straightforward due to the variety of
as the foregrounds have comparable intensity with the targeted possible CO masks and their nuanced effects on both CO and
signal and the dominating low-J line emission is scarcely dis- [CII] PS. To unravel these complex behaviors, we have adopted
tributed in the surveyed volume (i.e., their CV is high). On the a three-step approach: initially focusing on CO alone to deduce
other hand, for the recovery of the [CII] emission from z > 5 the optimal masking sequence and strategy (Sect. 4.2.1), then
galaxies, the situation appears less favorable as the foregrounds examining the impact of masking on the [CII] signal in isolation
are on average more than an order of magnitude brighter than the (Sect. 4.2.2), and finally analyzing the combined effects to fully
targeted signal and the dominating low-J line emission is homo- understand the interplay between masking, CO contamination,
geneously distributed in the surveyed volume (i.e., CV is low). and the [CII] signal (Sect. 4.2.3). This comprehensive approach
In the following section, we explore and quantify how these dif- will not only enable us to identify the optimal masking depth,
ferences impact the effectiveness of the masking technique in but also to assess the level of contamination at this depth.
removing the CO signal. There, it becomes clear that the differ-
ences in the CV of the CO intensity result in high masking ef- 4.2.1. Impact of masking on the CO PS
ficiency (significant CO reduction with a small fraction of the
survey volume masked) for our low-redshift (high-frequency) As already discussed in Sect. 3, for each frequency band and for
tomographies, but low masking efficiency (poor CO reduction each CO line we generated two distinct masking arrays: a "com-
with a large fraction of the survey volume masked) for our high- plete" masking array that masks all the galaxies in our external
redshift (low-frequency) tomographies. catalog with zJup = (νJup /νobs ± 20 GHz) − 1 and a "bright" mask-
ing array that among the galaxies of the "complete" subsample
masks only those with ∆MS > 0.
4.2. Evaluating the masking technique efficiency
Given the multitude of available masks, each with a unique
The effectiveness of a foreground removal technique can only impact on the observed data, identifying the optimal mask-
be evaluated by its impact on the statistic of interest. Our study ing sequence becomes a critical challenge in our analysis.
Article number, page 9 of 19
 A&A proofs: manuscript no. aanda

Masking the CO at k = 0.02 0.32 Mpc 1

106 106
CO CO
105 105 0 [CII]
0 3b
4b 3b 4b3c
5b 5b
6b 7b 8b 6b
7b 8b 5c
104 3c 104 4c

[CII]
k 3P/2 2 (Jy/sr)2

103 103
225 GHz 280 GHz
0 20 40 60 0 20 40 60

[CII]
106 106
[CII]
10 5 0
3b,3c 105 0
4b,4c CO
4b
5b 4c CO 5b 6b 7b 5c 6c
6b 7c
7b 8b 5c 6c

104 350 GHz 104 410 GHz

0 20 40 60 0 10 20 30 40
Percentage of Voxels Masked
Fig. 9: Effect of masking on the CO-only PS. The range of PS between the "low-contamination" and "high-contamination" models
(Sect. 2.3.2) at k = 0.02 − 0.32 Mpc−1 for CO lines (blue) is shown in relation to the percentage of voxels masked within the LIM
tomographies for the frequency ranges of 225 ± 40 GHz (associated with z[CII] = 7.4), 280 ± 40 GHz ( z[CII] = 5.8), 350 ± 40 GHz (
z[CII] = 4.3), and 410 ± 40 GHz ( z[CII] = 3.7). The lines represent the optimal CO masking sequence for our fiducial model, while
each point and label indicate the Jup and subsample ("b" for "bright" and "c" for "complete") associated with the masking array
applied. The same masking sequence is applied to the "low-contamination" and "high-contamination" models (Sect. 4.2.1). As an
illustration and assuming at this stage that the mask does not affect the [CII] PS (but see Sect. 4.2.2), we show the fiducial and range
of [CII] PS predictions in red.

Therefore, it is important to establish a reliable criterion for and white noise become increasingly important, it is not practi-
determining the most effective sequence of masks. The ef- cal to evaluate the optimal CO masking sequence directly from a
fectiveness, E(Jup , bright or complete) of masking arrays, realistic tomography including CO, [CII] and instrumental white
M(Jup , bright or complete), is quantified as, noise. However, for all our tomographies, we observed that the
Standard deviationunmasked − Standard deviationmasked optimal CO masking sequence does not vary much across the
E= . (6) range of CO foreground models implemented in our work. This
Number of masked voxels implies that the optimal CO masking sequences inferred here on
The reason for this definition of E is that we need a scale inde- our CO-only tomographies are applicable to first order to real
pendent value that correlates with the amplitude of the PS per observations. Any deviation from the "real" optimal sequence is
number of masked voxels, like the standard deviation of the in- unlikely to affect the result significantly.
tensity normalized by the number of voxels does.
The masking array with the highest E is applied iteratively to The optimal CO masking sequence as well as their impact on
the CO-only tomography, continuing until all arrays have been the CO PS as a function of masking depth is depicted in Fig. 9
utilized. With each iteration, we recalculate the effectiveness of at k = 0.02 − 0.32 Mpc−1 . By construction of our optimal CO
subsequent masks on the updated data. This iterative process, masking sequence, the slopes of the lines connecting the CO PS
performed on our CO-only simulations, identifies a sequence of as a function of masking depth starts by having negative values
masking arrays for each LIM tomography. It is important to note that get progressively less negative as the masking depth increase
that this optimal sequence of CO masks remains unchanged at and so does the masking effectiveness, E. This trend of progres-
350 ± 20 GHz and 410 ± 20 GHz if evaluated on a tomography sively less negative slope is, however, not always observed in this
including CO, [CII], and realistic white noise for the FYST LIM initial phase due to fact that the definition of E is scale indepen-
survey. In such tomographies, the masking efficiency (E) for dent and thus not tailored to the specific k ≈ 0.02 − 0.32 Mpc−1
each CO line is indeed still dominated by the CO signal. How- explored in Fig. 9. Overall, this figure demonstrates that in this
ever, at 225 ± 20 GHz and 280 ± 20 GHz, where the [CII] signal initial phase, the decrease of the CO PS with masking depth is
Article number, page 10 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

mostly driven at the high frequency bands by the masking of the [CII] PS Masking Bias, 350 GHz
CO Jup = 3 (for 350 ± 20 GHz) and Jup = 4 line (for 350 ± 20 0.30
GHz and 410 ± 20 GHz) which results in a 50% reduction of the [CII] Faint
[CII] Luminous
CO PS. At the low frequency bands (225 ± 20 GHz and 280 ± 20 0.25
GHz) the decrease is more gradual, yielding to a reduction of 41% masked
one order of magnitude by masking all Jup < 8 lines. 0.20
As the masking process intensifies, a critical threshold is

bmask
reached where the amplitude of the CO PS reaches a mini- 0.15
mum value (hereafter called the optimal masking depth) and af- 21% masked
ter which it rises despite the increasing masking depth. This oc- 0.10
curs when the CO line being masked contributes less to the to-
tal intensity of the tomography than the residual emissions from 7% masked
0.05
the brighter CO lines that have been masked previously. At this
point, the mask begins to function akin to a random mask on the 0.0 0.2 0.4 0.6 0.8
LIM tomography, as it is no longer targeting the brightest voxels. k (Mpc) 1
Moreover, since the intensity of the voxels follow a sparse, log-
normal distribution (Sect. 4.2.3), this random masking results
in the removal of more lower-intensity than high-intensity vox- Fig. 10: Masking bias representing the relative difference in the
els. However, the correction we apply to the PS for the survey PS between masked and unmasked [CII] tomographies for the
volume lost due to masking assumes that the intensity is uni- 350 ± 20 GHz frequency band. Comparisons are made for the
formly distributed throughout the survey volume. This assump- most pessimistic ("Faint," red lines) and optimistic ("Luminous,"
tion of uniformity, although simplistic, is necessary because the black lines) [CII] PS predictions for the masking depths of 7%,
non-homogeneity of the maps is model-dependent and cannot 21%, and 41%.
be accurately predicted without specific models. Nevertheless, it
results in an increase in the amplitude of the PS when the mask
does not follow the dominant galaxy population. PS signal ("Faint"; Fig 10). Then, we evaluated these biases for
From Fig. 9, we note that the optimal masking depth as we three masking scenarios, the "light" (7%), "moderate" (21%),
move to higher frequency tomographies is reached at progres- and "deep" (41%). In Fig. 10, we present these results for the
sively lower masking depths (60%, 40%, 40% and 30% for the 350 ± 20 GHz tomography.
225 GHz, 280 GHz, 350 GHz and 410 GHz accordingly). Com-
paring the amplitude of the CO PS at these optimal masking For the "light," "moderate," and "deep" scenarios, we find bi-
depths with that of the CII PS–assuming, at this stage, that it ases of ≈ 3%, ≈ 10%, and ≈ 22% at k = 0.02−0.32 Mpc−1 which
remains unaffected by the mask (but see Sect. 4.2.2 for further increase to ≈ 6%, ≈ 15%, and ≈ 28% at k = 0.62 − 0.92 Mpc−1 ,
discussion)–allows us to preliminarily evaluate the feasibility of respectively. The scale dependence of the masking bias depends
using this targeted masking approach to extract the CII PS sig- thus on the level of masking, and it intensifies as the masking
nal. We find that in all frequency bands except for the 225 ± 40 becomes more aggressive. At the limiting case of of a masking
GHz, the minimum fiducial CO PS is lower than the correspond- depth of 41%, bmask remains below 25% at large scales, but it can
ing [CII] PS. escalate to 30% at small scales. A similar scale-dependent bias
is observed by Van Cuyck et al. (2023) in their angular [CII] PS
predictions. This scale-dependent bias stems from the disparate
4.2.2. Impact of masking on the [CII] PS impact of bright sources on the [CII] PS amplitude at different
In the analysis carried out so far, we have assumed that masking scales. On larger scales, the PS is predominantly influenced by
does not affect the PS of the [CII] signal. In this section, we the clustering signal of numerous star forming galaxies, which
assess the validity of this assumption across different scales. itself is proportional to the first moment of the luminosity func-
By examining the influence of the masking on the [CII] PS tion (Paper I). However, as we move to smaller scales, the PS is
we deal with an issue already discussed in the context of the dominated by shot noise, which arises from the discrete nature
CO residuals (Sect. 4.2.1). Due to the sparsity of the signal, the of the luminous sources and is proportional to the second mo-
masking is artificially boosting the [CII] PS. Unlike the case of ment of the [CII] luminosity function (Paper I). At this scale the
the CO emission, where this effect becomes prominent only af- PS is thus strongly affected by the few, more luminous sources.
ter a certain threshold, boosting of the [CII] PS commences from The low probability of masking these rare bright sources leads to
the onset of the masking procedure as the [CII] emission is in- a significant masking bias due to volume corrections calculated
variably not correlated with the mask. This boosting of the [CII] assuming an homogeneous source distribution.
PS can henceforth be seen as a bias introduced by the masking Inverting the masking process to correct for masking bias,
process, that contributes additional uncertainty to our analysis. that is, deconvolving the masking array with the masked in-
We defined this masking bias, bmask , as the relative difference tensity maps, would require uncertain modeling of clustering
at a given scale between the PS amplitudes of the masked and components in order to extrapolate the [CII] emission of the
unmasked [CII] tomographies, that is, masked voxels. Furthermore, this approach could introduce ar-
tifacts when applied to a map with white noise, further com-
Pmasked − Punmasked plicating the PS analysis. However, it should be noted that this
bmask = . (7) scale-dependent, and sometimes significant, masking bias does
Punmasked
not vary greatly (<5%) depending on the exact [CII] model
We evaluated this scale-dependent bias using two extreme mod- used. This suggests that reliable scale-dependent corrections for
els for the [CII] tomography, that is, our model with the high- this masking bias can be efficiently calculated using the specific
est (hereafter "Luminous") and our model with the lowest [CII] masking strategies employed and [CII] models that are correct
Article number, page 11 of 19
 A&A proofs: manuscript no. aanda

to first order. Of course, forward model fits to the observed [CII] of around 15%). In contrast, at 225 ± 20 GHz and at optimal
PS will be able to account for this bias by construction. masking depth of 60%, the CO signal still dominates. In this
band, an alternative masking approach will be required.
From the observational point of view, the results of Fig. 11
4.2.3. Impact of masking on the CO-Contaminated [CII] PS
make clear that for a real dataset, the evolution of the PS as func-
We now turn our attention to assessing the impact of masking tion of the masking depth and the location of the optimal mask-
on the CO-contaminated [CII] PS, P(CO+[CII]) , specifically within ing depth provide key insights into the initial contamination lev-
the k = 0.02−0.32 Mpc−1 range (Fig. 11). The masks are applied els of the tomography. A very low value for the optimal mask-
with the optimal sequence inferred using the CO-only tomogra- ing depth (≲ 10%), associated with an amplified PS as we sur-
phies shown in Fig. 9 (Sect. 4.2.1). Building upon the analysis pass this threshold, unambiguously indicates the predominance
presented in Sect. 4.2.2, we include the bias introduced by mask- of [CII] in the initial tomography. In contrast, a value of the opti-
ing on the [CII] signal, which translates into a gradual increase in mal masking depth greater than ≳ 10% indicates the dominance
the [CII] PS as masking depth increases. Finally, as a last piece by CO foregrounds in the initial tomography. In this case, accu-
of information, the uncertainty associated with the [CII] PS for rately assessing the extent of CO contamination at the optimal
a 2000-hour FYST survey, denoted as σ[CII] , is plotted in Fig. 11 masking depth is challenging in a real dataset as no clear signa-
as well. This uncertainty, encompassing both instrumental white tures are visible in the PS and as the [CII] and CO emitters share
noise and sample variance, is defined in Paper I by the equation: similar spatial frequency characteristics, complicating their dif-
ferentiation in Fourier space.
Separation between the [CII] and CO emitters becomes
P[CII] (k, z) + PN clearer when analyzing how these populations contribute to
σ[CII] = √ . (8)
Nm (k) the average total power of the signal. Specifically, this can be
achieved by examining their contributions to the zero displace-
Assuming that the distribution of masked voxels is uniform, ment of the PS from constructing a histogram that represents the
this uncertainty increases with the masking depth proportionally distribution of squared voxel intensities, each weighted by its
to the decrease in the number of k-modes across all k-bins. A own squared intensity (Fig. 12). It is crucial to note, however,
Monte Carlo algorithm validated this uniformity assumption, es- that in order to create this histogram, knowledge of the absolute
tablishing a consistent relationship across all scales we exam- intensity of the tomography is needed, while this information is
ined: partially lost by our atmospheric and astrophysical continuum
foregrounds removal. To estimate a posteriori the magnitude of
s the mean intensity, we used the PS value at large scales, since
σ[CII]
r
Nk, not masked Nvoxels, not masked we demonstrated in Paper I that k3 Pclustering (k ≈ 10−2 Mpc−1 ) ≈
= = . (9)
σ[CII], masked Nk Nvoxels 2
Imean .
Equipped with this proxy for the mean intensity, we can gen-
By integrating all the elements presented in Fig. 11, we can erate a weighted histogram of the tomography, masked to the
assess whether the objective of masking has been achieved, that optimal depth. When this histogram displays two peaks instead
is, to reduce the amplitude of the CO PS below the measurement of one, it signifies a transition from a CO-dominated to a [CII]-
uncertainty of the [CII] PS (σ[CII] ), while ensuring that this un- dominated tomography, achieved by eliminating the brighter CO
certainty remains below the amplitude of the [CII] PS. Two dis- sources. Conversely, the presence of a single peak signifies that
tinct cases are evident in Fig. 11. Firstly, in the higher frequency the tomography remains CO-dominated, primarily by the less in-
bands (350 ± 20 and 410 ± 20 GHz), P(CO+[CII]) initially under- tense CO sources that were not masked in earlier iterations. Fig.
goes a slight decrease at masking depths of < 10%, attributed to 12 illustrates these two cases for the 280 ± 20 GHz tomogra-
the masking of the rare but bright Jup = 3 and Jup = 4 sources, phy. For the first case we considered our fiducial [CII] and CO
immediately followed by a gradual increase, mainly due to the PS whereas for the second we combined our fiducial [CII] with
[CII] PS masking bias (the CO PS masking bias only playing a our most pessimistic CO PS predictions. For the first case of the
role at masking depths > 20%). As already mentioned, since the fiducial CO PS, as we increase the masking depth, a secondary
[CII] emission is invariably uncorrelated with the mask, such an peak emerges at the brighter end of the histogram, correspond-
increase in PS from the onset of the masking procedure is char- ing to the population of [CII] emitters. Notably, at the optimal
acteristic of the dominance of the [CII] signal in the unmasked masking depth, these two peaks have similar heights, marking
tomography. At these high-frequency and at the optimal mask- the transition from CO dominance to [CII] dominance in the PS
ing depth of < 10%, the [CII] signal dominates and, more impor- of this masked tomography. This emergence and predominance
tantly, it is well above σCII . This guarantees accurate detection of of this secondary peak thus signifies the success of the masking
the [CII] PS by the FYST LIM survey in these bands. Secondly, process at the optimal masking depth. On the contrary, in the sec-
in the lower frequency bands (225 ± 20 and 280 ± 20 GHz), ond case, even at the optimal masking depth, the [CII] peak re-
P(CO+CII) decreases progressively with masking depth, reaches mains hidden in the bright tail of the CO peak. This single-peak
a minimum at the optimal masking depth of ≈ 40 − 60% and histogram, which moves only towards lower voxel intensities as
finally increases due to a combination of CO and [CII] mask- the masking depth increases, signifies the failure of the masking
ing biases. At 280 ± 20 GHz, the decrease in P(CO+CII) is less process at the optimal masking depth.
pronounced than at 225 ± 20 GHz, and in the former case, the Unfortunately, this rather simple method to distinguish
masked tomography effectively switches from CO-dominant to CO-dominated from CII-dominated tomography at the optimal
[CII]-dominant, whereas in the latter case, the masked tomogra- masking depth is strongly affected by the presence of white
phy remains CO-dominant. At 280 ± 20 GHz, and at the optimal noise. For example, considering the white noise of the plane
masking depth of 40%, the [CII] signal dominates and lies above 2000 hours of the FYST LIM survey, the peak(s) of the weighted
σCII . FYST LIM will thus detect, albeit marginally, the [CII] PS histogram is entirely hidden by it (Fig. 12). Nevertheless, with
in this band (signal-to-noise ratio of two and CO contamination five times deeper observations, the white noise distribution
Article number, page 12 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

Masking the CO at k = 0.02 0.32 Mpc 1

106 106
CO CO
105 105 3b0 4b3c 5b 6b 4c 5c
0 4b 3b 7b 8b
5b 6b 7b 3c 8b [CII] [CII]
104 [CII] 104
[CII]
k 3P/2 2 (Jy/sr)2

103 103
225 GHz 280 GHz
0 20 40 60 0 20 40 60

[CII]
7b 5c 6c
106 7c 10 6 0,4b,4c,5b 6b
8b 5c 6c
0,3b,3c,4b,5b,4c,6b 7b
[CII]
105 [CII] 105 [CII] CO
CO

104 350 GHz 104 410 GHz

0 20 40 60 0 10 20 30 40
Percentage of Voxels Masked
Fig. 11: Same plot as Fig. 9 but with the masking bias of [CII] taken into account. We also show the [CII] uncertainty (σ[CII] , white
line) and the total line (CO+[CII]) PS for the fiducial models (black line)

moves enough towards fainter voxels (Ivox 2

< 104 (Jy/sr)2 ) allow- the optimal masking depth and a novel indicator for successful
ing us to identify the emergence of the [CII]-related secondary CO foreground removal.
2
peak at Ivox > 104 (Jy/sr)2 . This white noise distribution effec- Our analysis revealed significant variations in CO contami-
tively acts as a threshold on the x-axis, below which observations nation across different frequency bands of the EoR-Spec, with
become indiscernible. notably lower contamination at higher frequencies. This vari-
In essence, our technique unfolds in two steps. We begin ation significantly impacts the efficiency of our masking tech-
by incrementally masking the LIM tomography until we iden- nique, which shows greater effectiveness at frequencies where
tify the point where the PS at k = 0.02 − 0.32 Mpc1 is mini- initial contamination is lower. Specifically, we found that the tar-
mized, revealing the optimal depth for masking. Following this, geted [CII] signal could be detected with minimal masking and
we examine the weighted voxel histogram of the tomography thus minimal masking bias at 410 GHz and 350 GHz (Fig. 13).
masked to this depth. In this histogram –which is calibrated us- In contrast, at 280 GHz and particularly at 225 GHz, our simple
ing the PS measurement and weighted by the square of the voxel CO masking technique faced challenges due to the high level of
intensities– we look for a bimodal distribution as confirmation foreground CO emissions, underscoring the need for alternative
of successful [CII] signal isolation. strategies at these frequencies.
Our work aligns with the efforts of Van Cuyck et al. (2023)
5. Discussion performed in the framework of the CONCERTO experiment,
which similarly aimed to test via simulation the extraction of
This study aimed to quantify the contamination of the the [CII] signal from the (post-)EoR universe using a compa-
(post-)EoR [CII] LIM signal by foreground CO emissions for rable masking strategy. Despite the challenges in making direct
the upcoming FYST LIM survey, utilizing post-processing of comparisons due to differences in angular and spectral resolu-
the TNG300 hydrodynamical simulation. The strength of our ap- tion, both studies highlight the limitations of masking strategies
proach lies in its capacity to generate a range of predictions that at high redshifts, especially beyond z ≥ 6.5. This consensus em-
account for the inherent uncertainties in CO luminosity functions phasizes the complexities of masking strategies and their vari-
(Riechers et al. 2019; Decarli et al. 2020), and for variations in able effectiveness across redshift. Diverging from Van Cuyck
the CO SLED of galaxies. Based on these predictions, we as- et al. (2023), we chose to exclude the [CI] line from our simu-
sessed the effectiveness of a masking technique that leverages lations due to its highly uncertain intensity values, aligning with
an external catalog to identify and subsequently mask bright CO methodologies like those suggested by Yue & Ferrara (2019).
emitters. This led to the identification of a method to determine The absence of [CI] contamination in our simulations is proba-
Article number, page 13 of 19
 A&A proofs: manuscript no. aanda

1e11 Successful case 1e13 Unsuccessful case

Total PS masked Total PS masked
Fitted Double Lognormal Fitted Double Lognormal
7 1.0

6
0.8
5
2

2
2 (Jy/sr)

2 (Jy/sr)
dex

dex
4 0.6
Ibin

Ibin
3
0.4
2
0.2
1

0 0.0
101 102 103 104 105 106 107 101 102 103 104 105 106 107
Ivox
2 (Jy/sr)2 Ivox
2 (Jy/sr)2

7 1e12
Successful Case 1e13 Unsuccessful Case
WN for 5x deeper survey 8
6 WN for 2000 hours
7 WN for 2000 hours
5 6
2

4 2 5
2 (Jy/sr)

2 (Jy/sr)
dex

dex 4
3
Ibin

Ibin
3
2
2 Optimal masking depth WN for 5x deeper survey
Optimal masking depth
1 1

0 1 0 1
10 102 103 104 105 106 107 10 102 103 104 105 106 107
Ivox
2 (Jy/sr)2 Ivox
2 (Jy/sr)2

Fig. 12: Pixel distribution histograms weighted with the voxel intensity squared, illustrating the distinction between successful (left;
fiducial CO) and unsuccessful (right; pessimistic CO) cases of CO masking of the FYST 280 ± 20 GHz tomography. The top panels
show the weighted voxel histograms of these 280 ± 20 GHz tomography masked to the optimal depth and fitted double lognormal
distributions. The lower panels detail the progressive masking depth, each curve representing the lognormal fits of the weighted
histogram at increasing depth. This highlights the emergence of a bimodal distribution in the successful case versus a single peak
distribution in the unsuccessful case. The lognormal fit to the weighted histogram of a white noise tomography equivalent to the
upcoming 2000 hours FYST LIM (black dashed line) and a fit to the white noise tomography of a survey 5 times deeper (gray
dashed line) are superimposed.

bly largely compensated by the very wide range of CO models, Our decontamination method itself comes with its limita-
including very pessimistic ones, taken into account in our study. tions. Despite its reliability, informed masking is not the most
The precise impact of [CI] lines, along with other foregrounds, efficient technique, particularly in terms of survey volume loss,
is expected to be clarified with the advent of first light science at low frequencies. The challenge remains to develop more re-
observations. fined techniques, that de-confuse the line PS without the cost
of losing survey volume. There are several mathematically ele-
gant techniques that could achieve this goal. One such approach
A notable limitation of our study, also highlighted by would be utilizing the wide frequency coverage of the EoR-
Gkogkou et al. (2023) and Van Cuyck et al. (2023), concerns Spec to measure the CO PS indirectly from the measurements
the impact of sample variance, of both the CO and [CII] emit- of the cross-correlation of three lines coming from the same
ters. Our reliance on the single realization of the TNG300 sim- galaxies, leveraging the statistical independence of the rest of
ulation introduces a potential limitation in terms of the accuracy the line-components. Another approach involves exploiting the
of our results. Nonetheless, given the wide range of models con- anisotropic nature of the 2D PS of interloper lines (Cheng et al.
sidered in our simulation, we assume that the influence of sam- 2016). Upon projection on the frame of reference of the targeted
ple variance on the effectiveness of our masking technique is line, the PS of these interloper lines exhibits distinct anisotropies
relatively minor. Nevertheless, recognizing that sample variance that can be used to differentiate between the target signal and the
could pose a significant challenge in the context of future for- foreground contamination. Both approaches are promising, yet
ward modeling of the [CII] LIM observations, we will explore they share the limitation of failing to recover the valuable Fourier
the development of multiple realizations of wide observational phase information, which is essential for analyzing histograms
cones by employing machine learning techniques (Garcia et al. related to voxel intensity distribution. A third approach that re-
2023).
Article number, page 14 of 19
 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

6. Conclusion
Intrinsic vs Masked [CII] PS, z[CII] = 4.1 4.8
Intrinsic Results Our study evaluated the level of contamination from CO rota-
107 Masked Results tional line emission when observing the (post-)EoR [CII] PS us-
ing the upcoming FYST LIM survey and presented a method
for mitigating it. We created mock tomographies of the CO line
k 3P/2 2 (Jy/sr)2

emitters by post-processing the TNG300 simulation, account-

ing for the inherent uncertainties in the observed CO luminosity
functions and the observed CO SLEDs. The resulting range of
106 CO predictions is in good agreement with the ASPECS observa-
tions.
Our analysis revealed that the total mean intensity of
7% of the survey volume masked CO emissions remains relatively constant, at approximately
1000 Jy/sr, across the observed frequency spectrum. This emis-
0.0 0.2 0.4 0.6 0.8 sion predominantly originates from low-J CO transitions (Jup <
k (Mpc) 1 8) in galaxies at lower redshifts (z < 2). In contrast, the [CII]
mean intensity exhibits a significant upward trend, increasing
Fig. 13: Comparison of the [CII] PS for [CII] observations in the from about 10 Jy/sr at 225 GHz to 2000 Jy/sr at 410 GHz, mak-
redshift range z[CII] = 4.1 − 4.8. The squares denote the com- ing the [CII] the dominant emission line at frequencies above
bined CO+[CII] PS after masking CO bright voxels, resulting in 364 GHz.
a 7% reduction in survey volume. The associated error bars on
the squares capture the uncertainty in [CII] measurements due to Based on these predictions, we assessed a foreground re-
white instrumental noise and sample variance. The blue crosses moval strategy that involves identifying and masking voxels con-
depict the intrinsic [CII] PS results, binned at the same ∆k. The taining bright CO emitters, utilizing mock external catalogs that
continuous line represents the intrinsic [CII] PS results but with exhibit realistic features. A key aspect of our analysis was the
a finer binning. determination of mask effectiveness, which we defined as the
ability of a mask to reduce the standard deviation of the inten-
sity of the signal, normalized by the number of voxels masked.
Through iterative application and reassessment of these masks
on CO-only tomography, we were able to refine our masking
captures the Fourier phase information is using conditional gen- sequence to achieve optimal foreground removal. The optimal
erative adversarial networks to de-confuse the signals at the map sequence for CO masks found here is broadly applicable to real
level rather than at the PS level. Moriwaki et al. (2020) focused observations, as we see very little variation into this sequence
on de-confusing two lines (Hα and OIII), using a large set of across the broad range of CO model considered here.
simulated data for training their models. Although this approach Our analysis also highlighted the frequency dependency of
is expected to be very effective, it is challenging to estimate the the optimal masking depth, which aligns with the point where
uncertainties involved in the de-confusion method of the mea- the PS of the masked tomography reaches its minimum. While
surements. Finally, it is worth noting the existence of a fourth less than 10% masking depth is required at higher frequencies
approach that is more robust against the model uncertainty and (350 ± 20 GHz and 410 ± 20 GHz), a deeper masking of more
still recaptures the phase information and that relies on the iden- than 40% is necessary at lower frequencies (280 ± 20 GHz and
tification and extraction of foreground sources that are charac- 225 ± 20 GHz).
terized by multiple interloping emission lines. This can be done
by fitting multiple SLED templates to each line-of-sight of the At these masking depths, according to our fiducial models,
tomography using the sparse approximation (Cheng et al. 2020). [CII] emission dominates and significantly exceeds the typical
Given the wide frequency range of the EoR-Spec, we consider instrumental white noise levels at higher frequencies, ensuring a
this method to be the most promising alternative approach be- robust detection of the [CII] PS at 350 ± 20 GHz and 410 ± 20
yond targeted masking. This approach will be tested and evalu- GHz. A tentative detection is also expected at 280 ± 20 GHz,
ated in the context of the FYST LIM survey on the third part of where [CII] marginally prevails over CO contamination. How-
our paper series. ever, at 225 ± 20 GHz, despite deep masking, CO contamination
continues to overshadow [CII] emission, highlighting the urgent
While line foregrounds present significant challenges, the need for advanced decontamination techniques.
importance of atmospheric and astrophysical continuum fore- While estimating the optimal masking depth for decontami-
grounds should not be overlooked. Although Van Cuyck et al. nating a real LIM tomography is relatively straightforward, dis-
(2023) successfully applied the asymmetric re-weighted penal- cerning the levels of decontamination achieved at this depth
ized least-squares technique to remove contamination from the presents a more complex challenge. This complexity arises be-
cosmic infrared background, reinforcing our focus on line in- cause CO and [CII] emitters share similar spatial frequency char-
terlopers as the primary challenge at low frequencies (225 ± 20 acteristics, making it difficult to distinguish between the two
GHz, 280 ± 20 GHz), the issue of atmospheric foregrounds re- solely based on their distribution in Fourier space. A practical
mains unresolved and may only be addressed through the use of solution involves creating a weighted voxel histogram, where
mock detector data streams. Unfortunately, it is at the high fre- each voxel is weighted by the square of its intensity and the his-
quencies (350±20 GHz, 410±20 GHz), where CO contamination togram is calibrated using the PS measurement of the tomogra-
is not problematic that we expect atmospheric contamination to phy masked at the optimal depth. Detailed examination of this
play a critical role. This juxtaposition makes atmospheric inter- histogram for signs of bimodality can reveal a successful tran-
ference potentially the primary barrier to detection at the two sition from a CO-dominated to a [CII]-dominated state. This
higher frequency bands of FYST LIM survey. method will, however, only be applicable to data with very low
Article number, page 15 of 19
 A&A proofs: manuscript no. aanda

instrumental white noise, about five times lower than that antic- Huber, Z. B., Choi, S. K., Duell, C. J., et al. 2022, in Millimeter, Submillime-
ipated for 2000 hours of observation with the EoR-Spec. ter, and Far-Infrared Detectors and Instrumentation for Astronomy XI, Vol.
12190, 121902H
Our study lays the groundwork for future LIM studies into Hunter, J. D. 2007, Computing in Science & Engineering, 9, 90
the high-redshift universe. The development of decontamination Kaasinen, M., Scoville, N., Walter, F., et al. 2019, ApJ, 880, 15
techniques capable of extracting the [CII] high-redshift signal Kamenetzky, J., Rangwala, N., & Glenn, J. 2017, MNRAS, 471, 2917
as well as the detailed modeling of the astrophysical and atmo- Karoumpis, C., Magnelli, B., Romano-Díaz, E., Haslbauer, M., & Bertoldi, F.
spheric foregrounds suitable for forward modeling are the nec- 2022, A&A, 659, A12
Kennicutt, Robert C., J. 1998, ARA&A, 36, 189
essary next steps. Kim, J. 2011, A&A, 531, A32
Acknowledgements. We thank Dongwoo Chung, Patrick Breysse, Yoko Okada, Kogut, A., Dwek, E., & Moseley, S. H. 2015, ApJ, 806, 234
Anirban Roy, Steve Choi, Eiichiro Komatsu, Thoma Nikola, Reinhold Schaaf, Kovetz, E., Breysse, P. C., Lidz, A., et al. 2019, BAAS, 51, 101
Ralf Antonius Timmermann, Kaustuv moni Basu, Vyoma Muralidhara, Maude Lagache, G., Cousin, M., & Chatzikos, M. 2018, A&A, 609, A130
Charmetant, Nicholas Battaglia, Gordon Stacey and other members of the CCAT Laigle, C., McCracken, H. J., Ilbert, O., et al. 2016, ApJS, 224, 24
science working group for detailed discussions about EoR-Spec; We are grate- Le Fèvre, O., Béthermin, M., Faisst, A., et al. 2020, A&A, 643, A1
ful to Matthieu Bethermin, Athanasia Gkogkou, Mathilde Van Cuyck, Desika Lehmer, B. D., Brandt, W. N., Alexander, D. M., et al. 2005, ApJS, 161, 21
Narayanan, Anthony Pullen, Sylvia Adscheid, Elena Marcuzzo, Prachi Khatri, Lenkić, L., Bolatto, A. D., Förster Schreiber, N. M., et al. 2020, AJ, 159, 190
Benedetta Spina, and Cristiano Porciani for the helpful discussions. This re- Madau, P. & Dickinson, M. 2014, ARA&A, 52, 415
search was carried out within the Collaborative Research Center 956 (project Magdis, G. E., Gobat, R., Valentino, F., et al. 2021, A&A, 647, A33
ID 184018867), sub-projects A1 and C4 and within the Collaborative Research Magnelli, B., Boogaard, L., Decarli, R., et al. 2020, ApJ, 892, 66
Center 1601 (project ID 500700252), sub-projects C3 and C6, funded by the Mashian, N., Sturm, E., Sternberg, A., et al. 2015, ApJ, 802, 81
Deutsche Forschungsgemeinschaft (DFG). This research made use of NASA’s Moriwaki, K., Filippova, N., Shirasaki, M., & Yoshida, N. 2020, MNRAS, 496,
Astrophysics Data System Bibliographic Services; Matplotlib (Hunter 2007); L54
Astropy, a community-developed core Python package for astronomy (Astropy Moriwaki, K. & Yoshida, N. 2021, ApJ, 923, L7
Collaboration et al. 2013); PlotDigitizer (http://plotdigitizer.sourceforge.net/). Narayanan, D. & Krumholz, M. R. 2014, MNRAS, 442, 1411
Parshley, S. C., Niemack, M., Hills, R., et al. 2018, in Ground-based and Air-
borne Telescopes VII, Vol. 10700, 1070041
Peebles, P. J. E. 1968, ApJ, 153, 1
Perot, A. & Fabry, C. 1899, ApJ, 9, 87
References Pillepich, A., Nelson, D., Springel, V., et al. 2019, MNRAS, 490, 3196
Aihara, H., Armstrong, R., Bickerton, S., et al. 2018, PASJ, 70, S8 Pillepich, A., Springel, V., Nelson, D., et al. 2018, MNRAS, 473, 4077
Aravena, M., Spilker, J. S., Bethermin, M., et al. 2016, MNRAS, 457, 4406 Ponthieu, N., Grain, J., & Lagache, G. 2011, A&A, 535, A90
Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, Popping, G., Pillepich, A., Somerville, R. S., et al. 2019, ApJ, 882, 137
A33 Pullen, A. R., Breysse, P. C., Oxholm, T., et al. 2023, MNRAS, 521, 6124
Barkana, R. & Loeb, A. 2001, Phys. Rep., 349, 125 Riechers, D. A., Boogaard, L. A., Decarli, R., et al. 2020, ApJ, 896, L21
Becker, G. D., Bolton, J. S., & Lidz, A. 2015, PASA, 32, e045 Riechers, D. A., Pavesi, R., Sharon, C. E., et al. 2019, ApJ, 872, 7
Becker, R. H., Fan, X., White, R. L., et al. 2001, AJ, 122, 2850 Roy, A., Valentín-Martínez, D., Wang, K., Battaglia, N., & van Engelen, A. 2023,
Bernal, J. L. & Kovetz, E. D. 2022, A&A Rev., 30, 5 ApJ, 957, 87
Béthermin, M., Fudamoto, Y., Ginolfi, M., et al. 2020, A&A, 643, A2 Schaerer, D., Ginolfi, M., Béthermin, M., et al. 2020a, A&A, 643, A3
Béthermin, M., Gkogkou, A., Van Cuyck, M., et al. 2022, A&A, 667, A156 Schaerer, D., Ginolfi, M., Béthermin, M., et al. 2020b, A&A, 643, A3
Scoville, N., Lee, N., Vanden Bout, P., et al. 2017, ApJ, 837, 150
Boogaard, L. A., Decarli, R., Walter, F., et al. 2023, ApJ, 945, 111
Seager, S., Sasselov, D. D., & Scott, D. 2000, ApJS, 128, 407
Boogaard, L. A., van der Werf, P., Weiss, A., et al. 2020, ApJ, 902, 109
Shannon, C. 1949, Proceedings of the IRE, 37, 10
Bouwens, R. J., Illingworth, G. D., Oesch, P. A., et al. 2015, ApJ, 803, 34
Silich, S., Tenorio-Tagle, G., Torres-Campos, A., et al. 2009, ApJ, 700, 931
Bouwens, R. J., Smit, R., Labbé, I., et al. 2016, ApJ, 831, 176
Silva, M., Santos, M. G., Cooray, A., & Gong, Y. 2015, ApJ, 806, 209
Bouwens, R. J., Smit, R., Schouws, S., et al. 2022, ApJ, 931, 160
Solomon, P. M. & Vanden Bout, P. A. 2005, ARA&A, 43, 677
Boylan-Kolchin, M., Weisz, D. R., Johnson, B. D., et al. 2015, MNRAS, 453, Stacey, G. J., Geis, N., Genzel, R., et al. 1991, ApJ, 373, 423
1503 Sun, G., Chang, T. C., Uzgil, B. D., et al. 2021, ApJ, 915, 33
Breysse, P. C., Kovetz, E. D., & Kamionkowski, M. 2015, MNRAS, 452, 3408 Sun, G., Moncelsi, L., Viero, M. P., et al. 2018, ApJ, 856, 107
Carilli, C. L. & Walter, F. 2013, ARA&A, 51, 105 Tacconi, L. J., Genzel, R., Saintonge, A., et al. 2018, ApJ, 853, 179
Cassata, P., Liu, D., Groves, B., et al. 2020, ApJ, 891, 83 Valentino, F., Daddi, E., Puglisi, A., et al. 2020, A&A, 641, A155
CCAT-Prime Collaboration, Aravena, M., Austermann, J. E., et al. 2023, ApJS, Vallini, L., Gallerani, S., Ferrara, A., Pallottini, A., & Yue, B. 2015, ApJ, 813, 36
264, 7 Van Cuyck, M., Ponthieu, N., Lagache, G., et al. 2023, A&A, 676, A62
Cheng, Y.-T., Chang, T.-C., Bock, J., Bradford, C. M., & Cooray, A. 2016, ApJ, Vavagiakis, E. M., Ahmed, Z., Ali, A., et al. 2018, in Millimeter, Submillimeter,
832, 165 and Far-Infrared Detectors and Instrumentation for Astronomy IX, Vol. 10708
Cheng, Y.-T., Chang, T.-C., & Bock, J. J. 2020, ApJ, 901, 142 (SPIE), 107081U
Chung, D. T., Breysse, P. C., Cleary, K. A., et al. 2022, ApJ, 933, 186 Venemans, B. P., Findlay, J. R., Sutherland, W. J., et al. 2013, ApJ, 779, 24
Clarke, J., Karoumpis, C., Riechers, D., et al. 2024, A&A, 689, A101 Vieira, J., Aguirre, J., Bradford, C. M., et al. 2020, arXiv e-prints,
CONCERTO Collaboration, Ade, P., Aravena, M., et al. 2020, A&A, 642, A60 arXiv:2009.14340
Conselice, C. J., Wilkinson, A., Duncan, K., & Mortlock, A. 2016, ApJ, 830, 83 Wang, T.-M. & Hwang, C.-Y. 2020, A&A, 641, A24
Daddi, E., Dannerbauer, H., Liu, D., et al. 2015, A&A, 577, A46 Wang, T.-M., Magnelli, B., Schinnerer, E., et al. 2022, A&A, 660, A142
Davé, R., Crain, R. A., Stevens, A. R. H., et al. 2020, MNRAS, 497, 146 Weaver, J. R., Kauffmann, O. B., Ilbert, O., et al. 2022, ApJS, 258, 11
De Looze, I., Cormier, D., Lebouteiller, V., et al. 2014, A&A, 568, A62 Wilson, D., Cooray, A., Nayyeri, H., et al. 2017, ApJ, 848, 30
Decarli, R., Aravena, M., Boogaard, L., et al. 2020, ApJ, 902, 110 Yue, B. & Ferrara, A. 2019, MNRAS, 490, 1928
Decarli, R., Walter, F., Aravena, M., et al. 2016, ApJ, 833, 70 Yue, B., Ferrara, A., Pallottini, A., Gallerani, S., & Vallini, L. 2015, MNRAS,
den Brok, J. S., Chatzigiannakis, D., Bigiel, F., et al. 2021, MNRAS, 504, 3221 450, 3829
Downes, D. & Solomon, P. M. 1998, ApJ, 507, 615 Zaroubi, S. 2013, Astrophysics and Space Science Library, Vol. 396, The Epoch
Dragone, C. 1978, AT T Technical Journal, 57, 2663 of Reionization, ed. T. Wiklind, B. Mobasher, & V. Bromm, 45
Elbaz, D., Daddi, E., Le Borgne, D., et al. 2007, A&A, 468, 33 Zel’dovich, Y. B., Kurt, V. G., & Syunyaev, R. A. 1969, Soviet Journal of Exper-
Euclid Collaboration, Desprez, G., Paltani, S., et al. 2020, A&A, 644, A31 imental and Theoretical Physics, 28, 146
Euclid Collaboration, Ilbert, O., de la Torre, S., et al. 2021, A&A, 647, A117 Zhou, X., Gong, Y., Deng, F., et al. 2023, MNRAS, 521, 278
Euclid Collaboration, Moneti, A., McCracken, H. J., et al. 2022, A&A, 658, Zou, B., Choi, S. K., Cothard, N. F., et al. 2022, in Millimeter, Submillimeter, and
A126 Far-Infrared Detectors and Instrumentation for Astronomy XI, Vol. 12190,
Fan, X., Strauss, M. A., Richards, G. T., et al. 2006, AJ, 131, 1203 121902B
Garcia, K., Narayanan, D., Popping, G., et al. 2023, arXiv e-prints; ApJ, in press,
arXiv:2311.01508
Gkogkou, A., Béthermin, M., Lagache, G., et al. 2023, A&A, 670, A16
Harikane, Y., Inoue, A. K., Mawatari, K., et al. 2022, ApJ, 929, 1

Article number, page 16 of 19

 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

Appendix A. Impact of masking on the PS of

individual CO lines
This appendix presents the PS of individual CO (Jup = 3 − 8)
spectral lines in the frame of reference of the [CII] line at scales
of k = 0.02 − 0.32 Mpc−1 as a function of the percentage of the
survey volume masked (Fig. A1 and A2) following the optimal
masking sequence described in Sect. 4.2. For each CO line PS,
two distinct behaviors are observed during the masking process.
When the masking step specifically targets the spectral line in
question, there is a sharp decrease in the PS. Conversely, when
the masking step targets a different spectral line, a slow increase
in the PS is observed due to the masking bias (see Sect. 4.2.2).
The behavior of the PS during masking is influenced by how
the galaxies emitting the lines are distributed within the 3D to-
mography. Lines from galaxies that are highly localized within
the survey volume, such as CO(3-2) at 350 GHz, exhibit a sharp
decrease in their PS when masking targets them. Additionally,
there is almost no increase in PS when masking targets other
lines due to the minimal residuals left after masking. In con-
trast, lines from galaxies more evenly distributed throughout the
survey volume, such as CO(4-3) at 225 GHz, show a less pro-
nounced decrease in PS when masked, but a stronger increase
when other lines are masked, because there remains a substantial
amount of residual emission that gets amplified. A key indicator
of how uniformly the intensity of each line is distributed in the
tomography is the CV of its intensity, as shown in Fig. 8 and
described in detail in Sect. 4.1.

Article number, page 17 of 19

 A&A proofs: manuscript no. aanda

Fig. A1: Effect of masking on the PS of the brightest individual CO lines (Jup = 3 − 8). The lines represent our fiducial model, with
the same optimal masking sequence applied as in Fig. 9 and Fig. 11. The results are plotted for the frequency ranges 225 ± 40 GHz
(corresponding to z[CII] = 7.4) and 280 ± 40 GHz (z[CII] = 5.8)

Article number, page 18 of 19

 C. Karoumpis et al.: [CII] line intensity mapping the epoch of reionization with the Prime-Cam on FYST

Fig. A2: Same as Fig. A1, but for the frequency ranges 350 ± 40 GHz (z[CII] = 4.3) and 410 ± 40 GHz (z[CII] = 3.7).

Article number, page 19 of 19