How to Manage and Optimize Your Enterprise Data Integration Infrastructure

0 likes0 views

FME Flow is the powerful engine in your enterprise service infrastructure and supplies your employees, customers and applications with the data they need. Continuous monitoring and analysis of this business-critical component is therefore essential for stable operation and reliable data-driven decisions. In this talk con terra will show you how you can: -Ensure the stable operation of your FME Flow infrastructure. -Ensure the technical reliability of your FME processes. - Improve the operation and performance of your data integration infrastructure. In addition to FME Flow, you can also monitor other components of your enterprise infrastructure, such as ArcGIS Enterprise at the same time.

Be at the pulse of
your data
- How to manage and optimize your
enterprise data integration infrastructure
con terra
The Peak of Data
and AI 2025

2025
The
Peak
of
Data
and
AI
Benjamin Quest
Technical Sales
& Partner Manager
con terra

1. Introduction: Blood
2. Diagnosis
3. Recipe
4. Cure
Agenda

2025
The
Peak
of
Data
and
AI
If data is the blood of your organisation,
then FME Flow is the heart …

2025
The
Peak
of
Data
and
AI
… it powers your processes, your
decisions, your business!

2025
The
Peak
of
Data
and
AI
Data Streams On-Premises
Business Users
Decision Makers
Customers
Data Integration
Application Integration
Real-Time Data
Data Quality
Automation
GIS
Enterprise Systems
Cloud

2025
The
Peak
of
Data
and
AI
But what if it strokes?

At con terra we are no doctors, but
we are experts in system
integration and focused on
optimizing and monitoring your
processes and your infrastructure.

2025
The
Peak
of
Data
and
AI
Strategies to assure operational
success!
• Keep track on how your FME Flow performs from an external viewpoint
• Get transparency on operational data and use it
• Get insights and plan to improve

2025
The
Peak
of
Data
and
AI
service.monitor for FME
• Continously check your FME Flow for stable
execution
• Provide centralized access to „all data“
• Offer insight gaining statistics and diagrams
• Visualize Engine occupation
• Get notiﬁed on any event

2025
The
Peak
of
Data
and
AI
Be at the pulse of your
organisation
● Monitoring
● Analytics & usage behaviour
● Logging & operational data
Monitorin
g
Analytics Logging

2025
The
Peak
of
Data
and
AI
Monitoring
● Operational view point
○ Uptime monitoring
○ Engines registered and available
● Functional view point
○ Job has been executed like expected and scheduled
○ Log data is stating a deﬁned pattern

2025
The
Peak
of
Data
and
AI
Logging & operational data
● Powerful full text search on all log data
● Quickly investigate on failed jobs
● Understand overall server situation
● Get short term and long term data on job execution and engine performance

2025
The
Peak
of
Data
and
AI
Analytics and usage behavior
● Understand
○ the overall usage of your FME Flow infrastructure
○ which workspaces fail often or start to fail regurarly
● Solve
○ Conﬂicts and interdependencies of your job execution
● Plan
○ Better occupation of your FME Engines

2025
The
Peak
of
Data
and
AI
How do we do this?

2025
The
Peak
of
Data
and
AI
Recap:
● Understand the
importance of FME Flow
for your daily business
● Ask yourself: What can
be done to improve?
● service.monitor is the
solution to success.

2025
The
Peak
of
Data
and
AI
“A healthy FME Flow in a
healthy organisation.”
— “mens sana in corpore sano”
Roman poet Juvenal, around the end of the ﬁrst century AD

2025
The
Peak
of
Data
and
AI
ThankYou
Benjamin Quest
con terra
b.quest@conterra.de

Harness the full potential of AI with FME: From creating high-quality training data to optimizing models and utilizing results, FME supports every step of your AI workflow. Seamlessly integrate a wide range of models, including those for data enhancement, forecasting, image and object recognition, and large language models. Customize AI models to meet your exact needs with FME’s powerful tools for training, optimization, and seamless integration

From Field to Digital Twin: Leveraging FME for Efficient Data Ingestion in Di...Safe Software

Effective disaster recovery and damage mitigation require the ability to rapidly identify sensitive and vulnerable environmental elements. In crisis scenarios, swift and informed decision-making is paramount to minimizing impacts and protecting resources. This showcase highlights the implementation of FME as a critical data orchestration platform, enabling the seamless integration, organization, and processing of information gathered via a mobile web app during emergency phases. By delivering processed data to the digital twin in near-real-time, FME ensures that decision-makers have access to up-to-date, actionable insights precisely when they are needed most. This approach significantly enhances the speed and effectiveness of response strategies, ultimately improving outcomes in emergency management and environmental protection.

Bringing Your Data to Life: Unreal Engine & FME IntegrationSafe Software

Breaking Language Barriers in FME Workspaces with AISafe Software

Working at Tensing I get the opportunity to collaborate with some amazing colleagues in The Netherlands and France. They shared with me a range of impressive Workspaces which I needed to adapt for projects I am working on in the UK. Unfortunately, my Dutch and French language skills aren't quite up to scratch. That's when I thought there must be a way to use FME to translate the bookmarks and annotations in the workspaces so I could understand what each part of the process was doing. This presentation will focus on the use of FME and Generative AI able to create a solution to successfully translate bookmarks and annotations in a workflow. Leveraging the unique capabilities of the FME Platform to read in existing workspaces and make requests to AI to translate workspace components. I’ll explore the choices that were made along the way, providing an insight into the workflows and processes involved. Hopefully inspiring you to think outside the box of fun and innovative ways you might be able to use FME and AI in your daily tasks.

Operational Efficiency Through Intelligent Robotic Process AutomationSafe Software

Northern Ireland Water's Intelligent Operations Centre and integration of Robotic Process Automation (RPA) have revolutionized their workflows. By addressing critical operational challenges with automation, they have not only improved response times but also significantly reduced manual workloads, allowing their workforce to focus on value-added activities. The success of these initiatives has laid a solid foundation for further automation and digital transformation in the future.

Tackling Memory Challenges in FME When Working with Large OpenStreetMap DatasetsSafe Software

OpenStreetMap (OSM) is an open geographic database widely used in all kinds of tools and applications. Given the open-source nature of OSM, it contains an extensive range of information and geometries. However, it’s not feasible to use this data as-is unless you transform it to the data schema the client has in mind. This way we can create a customized dataset that can be downloaded by end users. In this project, we use FME for the conversion of the raw OSM data to a dataset that will be used as the foundational layer for specialized maps, such as walking or cycling maps. In a first step, we need to download OSM data. In this case we use Geofabrik, a trusted source for OSM extracts. This raw data, covering five European countries, needs mapping and filtering to meet the specific project requirements. Due to the large size of the dataset, memory limitations in FME pose significant challenges, leading to performance issues (e.g. crashes). A key part of the presentation focuses on these challenges and practical strategies for optimizing FME workflows to prevent memory overload, including methods for chunking data, reducing memory usage with specific transformers, and so on. Finally, I will introduce a FME flow app that enables users to easily download specific bounding boxes of cleaned and processed OSM data. By the end of this session, attendees will gain valuable insights into working with OSM data, handling memory limitations and creating easy interfaces for users.

Automating ArcGIS Content Discovery with FMESafe Software

Managing ArcGIS Enterprise can be overwhelming, not just due to the vast content that accumulates within an organization, but also because of the intricate web of connections between them. Backed by input from ArcGIS users and first-hand experience with this problem, I have developed a single-page application powered by FME to streamline this task. All with no additional infrastructure! This app serves as a centralized catalog that organizes all elements of an ArcGIS implementation in an intuitive, easy-to-navigate dashboard. Deployed via an FME Flow Project, it requires minimal setup and uses the ArcGIS REST API to automatically discover and map relationships between content, users, and groups. FME transforms repetitive tasks into efficient, automated workflows that wouldn’t be feasible otherwise. A key feature of the app is its one-click reporting, enabled by FME Flow webhooks, which generates instant, visual summaries. This project integrates ArcGIS REST API, images, JSON, HTML, and D3 for a robust and user-friendly content management solution. This presentation will showcase FME’s role in automating content discovery and managing relationships within ArcGIS. A live demo will highlight how this solution reduces human error, saves hundreds of hours and demonstrates the ease of deployment via FME Flow Projects. Already in use by several clients, this application empowers organizations, helping them save time and money and maximize their investment in FME and ArcGIS.

Smarter Aviation Data Management: Lessons from Swedavia Airports and SwecoSafe Software

Managing airport and airspace data is no small task, especially when you’re expected to deliver it in AIXM format without spending a fortune on specialized tools. But what if there was a smarter, more affordable way? Join us for a behind-the-scenes look at how Sweco partnered with Swedavia, the Swedish airport operator, to solve this challenge using FME and Esri. Learn how they built automated workflows to manage periodic updates, merge airspace data, and support data extracts – all while meeting strict government reporting requirements to the Civil Aviation Administration of Sweden. Even better? Swedavia built custom services and applications that use the FME Flow REST API to trigger jobs and retrieve results – streamlining tasks like securing the quality of new surveyor data, creating permdelta and baseline representations in the AIS schema, and generating AIXM extracts from their AIS data. To conclude, FME expert Dean Hintz will walk through a GeoBorders reading workflow and highlight recent enhancements to FME’s AIXM (Aeronautical Information Exchange Model) processing and interpretation capabilities. Discover how airports like Swedavia are harnessing the power of FME to simplify aviation data management, and how you can too.

Leveraging FME for Enhanced Decision Making in Offshore OperationsSafe Software

In offshore operations, on-the-fly decision making is crucial, no matter how well you plan. During a recent campaign, we leveraged FME to integrate data directly on the vessel, facilitating critical decision making, and allowing for adaptive and efficient data collection. By running workbenches, we streamlined data integration from various sensors and systems, enriched datasets, created spatialized reports, geocoded videos, and visualized data effectively. This approach significantly improved our situational awareness and operational efficiency, demonstrating the value of FME in complex offshore environments.

9-1-1 Addressing: End-to-End Automation Using FMESafe Software

This session will cover a common use case for local and state/provincial governments who create and/or maintain their 9-1-1 addressing data, particularly address points and road centerlines. In this session, you'll learn how FME has helped Shelby County 9-1-1 (TN) automate the 9-1-1 addressing process; including automatically assigning attributes from disparate sources, on-the-fly QAQC of said data, and reporting. The FME logic that this presentation will cover includes: Table joins using attributes and geometry, Looping in custom transformers, Working with lists and Change detection.

Green Light to a New Transportation Network Data with FME and AISafe Software

From Manual to Auto Searching- FME in the Driver's SeatSafe Software

Finding a specific car online can be a time-consuming task, especially when checking multiple dealer websites. A few years ago, I faced this exact problem while searching for a particular vehicle in New Zealand. The local classified platform, Trade Me (similar to eBay), wasn’t yielding any results, so I expanded my search to second-hand dealer sites—only to realise that periodically checking each one was going to be tedious. That’s when I noticed something interesting: many of these websites used the same platform to manage their inventories. Recognising this, I reverse-engineered the platform’s structure and built an FME workspace that automated the search process for me. By integrating API calls and setting up periodic checks, I received real-time email alerts when matching cars were listed. In this presentation, I’ll walk through how I used FME to save hours of manual searching by creating a custom car-finding automation system. While FME can’t buy a car for you—yet—it can certainly help you find the one you’re after!

Using the SQLExecutor for Data Quality Management: aka One man's love for the...Safe Software

The SQLExecutor is one of FME’s most powerful and flexible transformers. Pivvot maintains a robust internal metadata hierarchy used to support ingestion and curation of thousands of external data sources that must be managed for quality before entering our platform. By using the SQLExecutor, Pivvot can efficiently detect problems and perform analysis before data is extracted from our staging environment, removing the need for rollbacks or cycles waisted on a failed job. This presentation will walk through three distinct examples of how Pivvot uses the SQLExecutor to engage its metadata hierarchy and integrate with its Data Quality Management workflows efficiently and within the source postgres database. Spatial Validation –Validating spatial prerequisites before entering a production environment. Reference Data Validation - Dynamically validate domain-ed columns across any table and multiple columns per table. Practical De-duplication - Removing identical or near-identical well point locations from two distinct source datasets in the same table.

Using FME to Advance Roadway Safety and Capital PlanningSafe Software

Canadian municipalities are working towards Vision Zero goals to eliminate traffic collisions and make roads safer for everyone. To support this process, GHD created an FME solution that evaluates and assesses the inventory and condition of existing roadside safety systems and unprotected hazards using ArcGIS Feature Service data. This solution also generates remediation measures and capital cost estimates aligned with municipal requirements. With this information, municipalities can prioritize the most critical areas and plan their budgets more effectively to achieve safer roads. This presentation will demonstrate how FME can help to automate a transportation engineering data process to enhance decision-making and infrastructure planning for safer communities.

AI Agents and FME: A How-to Guide on Generating Synthetic MetadataSafe Software

Data Governance & Quality Supported by FMESafe Software

Connecting Data and Intelligence: The Role of FME in Machine LearningSafe Software

In this presentation, we want to explore powerful data integration and preparation for Machine Learning. FME is known for its ability to manipulate and transform geospatial data, connecting diverse data sources into efficient and automated workflows. By integrating FME with Machine Learning techniques, it is possible to transform raw data into valuable insights faster and more accurately, enabling intelligent analysis and data-driven decision making.

Enhancing Landscape Recolonization Assessment Through FME and Big Data Integr...Safe Software

In collaboration with the Federal Environment Agency, the Federal Agency for Cartography and Geodesy of Germany is using FME to simplify the assessment of landscape recolonization potential for Germany. As biodiversity declines, identifying refugial areas is crucial, and land cover plays a key role. Using vast amounts of high-resolution remote sensing data and national datasets, FME automates this complex Big Data process. Users just select their area of interest, and FME Flow gathers the data, performs land cover classification, and delivers a detailed map and stats to evaluate recolonization potential—all with just a few clicks.

What Location Data Does Google Have on You?Safe Software

Building Geospatial Data Warehouse for GIS by GIS with FMESafe Software

Data warehouses are often considered the backbone of data-driven decision-making. However, traditional implementations can struggle to meet the iterative and dynamic demands of GIS operations, particularly for Iterative Cross-Division Data Integration (ICDI) processes. At WVDOT, we faced this challenge head-on by designing a Geospatial Data Warehouse (GDW) tailored to the needs of GIS. In this talk, we’ll share how WVDOT developed a metadata-driven, SESSI-based (Simplicity, Expandability, Spatiotemporality, Scalability, Inclusivity) GDW that transforms how we manage and leverage spatial data. Key highlights include: - Simplifying data management with a lean schema structure. - Supporting ad hoc source data variations using JSON for flexibility. - Scaling for performance with RDBMS partitioning and materialized views. - Empowering users with FME Flow-based self-service metadata tools.We’ll showcase real-world use cases, such as pavement condition data and LRS-based reporting, and discuss how this approach is elevating GIS as an integrative framework across divisions. Attendees will walk away with actionable insights on designing GIS-centric data systems that combine the best of GIS and data warehouse methodologies.

Streamlining CI/CD with FME Flow: A Practical GuideSafe Software

Using FME to Help the Field Help You - Peak of Data & AISafe Software

My talk will focus on how I'm using a few workbenches that are run daily in FME Flow ensure that data is ready for final processing and integration into systems of record at bp Terminals and Pipelines. We have a few hundred field staff that use ESRI's Survey123 to collect data on repairs made to our pipelines. My talk will highlight how FME helps the GIS team perform spatial QC on that data, manage a complex review and approval process involving field and office staff, and make final loading of data into systems of record a breeze. The first workbench I'll highlight looks at the GPS point captured for the report, and whether it is within a certain distance of our pipeline centerline. If it is not within a certain distance, then the user will be alerted, and asked whether our centerline is in the wrong spot, or if their GPS point is in the wrong location. The second workbench I'll highlight will show how data collected in the field is moved through a review/approval process from field to office staff. FME takes data collected in Survey123, visualizes that data in a more human-readable format (Word Document), and alerts users when it is time for them to review and approve data assigned to them. Finally, I'll highlight how FME helps the GIS staff easily bulk load this data collected in the field into a PODS database for company-wide visualization and access.

Digital Transformation: Automating the Placement of Medical InternsSafe Software

Discover how the Health Service Executive (HSE) in Ireland has leveraged FME to implement an automated solution for its “Order of Merit” process, which assigns medical interns to their preferred placements based on applications and academic performance. This presentation will showcase how FME validates applicant data, including placement preferences and academic results, ensuring both accuracy and eligibility. It will also explore how FME automates the matching process, efficiently assigning placements to interns and distributing offer letters. Beyond this, FME actively monitors responses and dynamically reallocates placements in line with the order of merit when offers are accepted or declined. The solution also features a self-service portal (FME Flow Gallery App), enabling administrators to manage the entire process seamlessly, make adjustments, generate reports, and maintain audit logs. Additionally, the system upholds strict data security and governance, utilising FME to encrypt data both at rest and in transit.

Managing City of Cockburn's Cloud Migration and ESRI Field Operations with FMESafe Software

The City of Cockburn, located 20km south of Perth, is home to 130,000 people, expected to grow to 170,000 by 2026. It offers various attractions and services, including wetlands, employment opportunities, and community facilities. The city uses TechnologyOne ERP and ESRI ArcGIS for managing tasks and requests. Facing a November 2024 deadline to migrate TechnologyOne ERP to the cloud, the city needed to update its SQL and Python-based integrations with ArcGIS. The original developers had left, and the new team found the existing integrations difficult to manage. To address this, the city chose Safe Software’s FME for integration, with support from 1Spatial. They developed a Proof of Concept (PoC) for waste management integration and provided tailored FME training to the GIS and Business Systems teams. Initially using sFTP for data exchange, they switched to RESTful Web Services for more robust updates, utilizing FME’s Change Detector for real-time synchronization. The PoC was completed in two months, and all integrations within six months, ensuring a smooth transition to the cloud. This modernization allows the city to maintain and develop streamlined data flows between systems, ensuring efficiency and scalability for future need.

University Campus Navigation for All - Peak of Data & AISafe Software

Navigating around the 1000 acres of UBC Vancouver’s campus may look like a walk in a park, but when you’re a student in a wheelchair trying to find a powered door to the class you’re late for, it is not all that scenic. UBC’s GIS team will share the journey from the original 2003 static map application to the current web application built using FME and the ArcGIS JavaScript API. There were two main goals with this new application: 1. Enable the UBC community to easily interact with and navigate the campus by walking, cycling and accessible means (which include mobility constraints such as steep slopes, stairs, and doors that are not powered). 2. Provide high quality geospatial data that is authoritative and updated using FME to reflect existing conditions on campus including road, sidewalk or building closures for construction. We will cover the unique challenges in creating a navigation application which offers features that Google Maps cannot at the scale of a University campus. Door to door navigation, avoiding stairs and steep slopes, navigating to accessible entrances and up to date construction barriers are some of the features we will discuss.

Automating FME Flow Usage Statistics and AdministrationSafe Software

This presentation demonstrates how FME Form, FME Flow, PostgreSQL, an ArcGIS Dashboard, and Grafana were used to create interactive visualizations for monitoring job statistics and server performance. By leveraging the FME Flow API alongside Azure and an internal company API, we also automated FME Flow user administration. The same concepts from these FME workflows could be used to manage other systems with FME, such as ArcGIS Enterprise.

Automated Migration of ESRI Geodatabases Using XML Control Files and FMESafe Software

Efficient data migration is a critical challenge in geospatial data management, especially when working with complex data structures. This presentation explores an automated approach to migrating ESRI Geodatabases using FME and XML-based control files. A key advantage of this method is its adaptability: changes to the data model are seamlessly incorporated into the migration process without requiring modifications to the underlying FME workflow. By separating data model definitions from migration logic, this approach ensures flexibility, reduces maintenance effort, and enhances scalability.

Modernizing Vermont's Cyanobacteria Tracking ProgramSafe Software

The Cyanobacteria Tracking Program in Vermont is a collaborative effort between the Departments of Health (VDH) and Environmental Conservation (DEC), and the Lake Champlain Committee. Since 2003, trained volunteers, staff, and citizens have regularly monitored Vermont's bays, lakes, and ponds during the cyanobacteria season. This crucial information is shared with the public through the state's Cyanobacteria Tracker Map. In 2024, with new FME Flow licensing, the VDH GIS and Data Viz team overhauled a critical component: real-time data synchronization and the notification system. Using FME Flow, we can now monitor reports in near real-time, alert staff of new blooms, and flag potential errors in the data. Several enhancements have also been implemented this season. We can now monitor blooms near drinking water intakes and notify DEC staff. Another exciting enhancement is the integration with our Cooling Sites map, an interactive tool that helps the public find places to cool down on hot days. This map features cool indoor locations and parks with swimming access. However, some beaches may be closed due to blooms or elevated E. coli levels. Thanks to FME, we can now integrate these key pieces of information and update our Cooling Sites map with beach closure information in real-time. FME has made this real-time data integration not only possible but also easy to implement and manage. We are excited to continue modernizing our data systems using FME.

FIDO Alliance Seminar State of Passkeys.pptxFIDO Alliance

FIDO Seminar: New Data: Passkey Adoption in the Workforce.pptxFIDO Alliance

More Related Content

More from Safe Software (20)