Linking Scientific Instruments and Computation
Download as pptx, pdf0 likes46 views
The document discusses advancements in scientific instruments and computational techniques, emphasizing the need for modular and extensible systems to handle the increasing complexity and volume of data generated by new sensors. It highlights the development of flows for tasks such as data collection, analysis, and visualization, which have significantly accelerated processes like protein structure determination. Additionally, the document calls for collaboration to enhance these methods and expand their application across various scientific fields.
1 of 28
Download to read offline
Ad
Recommended
Working with Instrument Data (GlobusWorld Tour - UMich)
Working with Instrument Data (GlobusWorld Tour - UMich)Globus This document discusses using Globus to automate the management and analysis of large scientific instrument data. It provides examples of challenges with managing large datasets from the Event Horizon Telescope and applying Globus services and automation to help address these challenges. Specific use cases discussed include building connectomes from microscopy data and applying deep learning to flag bad scanning electron microscope images. The document emphasizes that automation needs transparency, results need to be easily findable, and leveraging specialized services can help.
Discovery Engines for Big Data: Accelerating Discovery in Basic Energy Sciences
Discovery Engines for Big Data: Accelerating Discovery in Basic Energy SciencesIan Foster The document discusses the development of discovery engines for big data to enhance research in basic energy sciences by integrating experiments, simulations, and data analytics. It emphasizes the importance of understanding disordered structures in materials science and presents strategies for accelerating discovery through automation and advanced computational methodologies. The aim is to improve knowledge management and collaboration across various facilities to better manage and analyze large datasets in energy science research.
Opportunities for X-Ray science in future computing architectures
Opportunities for X-Ray science in future computing architecturesIan Foster The document discusses advancements in computing architecture, highlighting trends such as petascale and cloud computing, with implications for x-ray science and facilities. It emphasizes the importance of massive parallelism, large data handling, and economic aggregation in future computing challenges. The document also touches on various computational methods and the necessity for automation in scientific processes to enhance discovery and efficiency.
Shaping the Future: To Globus Compute and Beyond!
Shaping the Future: To Globus Compute and Beyond!Globus This document introduces funcX, a new service for managing remote computation similar to how Globus manages data transfers. FuncX allows users to register Python functions and execute them on remote resources through a simple interface. It aims to make remote computing as easy as transferring files by handling authentication, resource configuration, and moving code and results between systems. The document demonstrates how funcX is being used for scientific applications like statistical model fitting and inverse spectroscopy that require running thousands of tasks on HPC resources. It solicits feedback to help guide the funcX roadmap and identifies common use cases around scaling jobs, automating workflows, and enabling community access through shared endpoints and functions.
Data Automation at Light Sources
Data Automation at Light SourcesIan Foster The document discusses data automation at light sources, highlighting the challenges posed by immense data rates and the limitations of current methods. It emphasizes the need for new experimental designs, data management and analysis techniques, proposing automation and outsourcing strategies to handle data effectively. Examples include high-speed storage services, cloud-based automation, and the Data and Learning Hub for managing machine learning models and data publishing.
Globus Labs: Forging the Next Frontier
Globus Labs: Forging the Next FrontierGlobus The document discusses challenges in research data management and analysis, highlighting issues such as inefficiencies, errors, and collaboration difficulties within large-scale science. It presents various tools and platforms developed by Globus Labs, including FuncX, Parsl, Polyner, and Ripple, aimed at automating research workflows and enhancing data accessibility and usability. The document emphasizes the importance of high-performance computing and the integration of distributed resources to optimize scientific analysis and improve research outcomes.
ECP Application Development
ECP Application Developmentinside-BigData.com The Exascale Computing Project focuses on developing applications for exascale hardware to enhance scientific and engineering missions. It encompasses diverse categories such as energy, chemistry, earth sciences, and data analytics, with over 50 projects designed to address specific challenges in these fields. Aiming for improved performance and efficiency, the project emphasizes early adoption of exascale technology to track progress and mitigate risks.
Sgg crest-presentation-final
Sgg crest-presentation-finalmarpierc The document provides an overview of the Science Gateway Group at Indiana University. It introduces the group members and describes their focus areas as developing open source software for cyberinfrastructure like Apache Rave and Apache Airavata. It discusses the group's work on extending collaborations with application scientists in various domains. The document also outlines possibilities for collaboration with the PTI CREST Lab on topics like scientific workflows and generalized execution frameworks.
Computation and Knowledge
Computation and KnowledgeIan Foster The document discusses how computation can accelerate the generation of new knowledge by enabling large-scale collaborative research and extracting insights from vast amounts of data. It provides examples from astronomy, physics simulations, and biomedical research where computation has allowed more data and researchers to be incorporated, advancing various fields more quickly over time. Computation allows for data sharing, analysis, and hypothesis generation at scales not previously possible.
Better Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumIan Foster This document discusses the computing continuum and efforts to enable better information faster through computation. It provides examples of how techniques like executing tasks closer to data sources or on specialized hardware can significantly accelerate applications. Programming models and managed services are explored for specifying and executing workloads across diverse infrastructure. There are still open questions around optimizing networks, algorithms, and applications for the computing continuum.
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...Globus Gladier is a framework that combines instruments, storage, and compute using loosely coupled services to automate and simplify data-intensive experimental research workflows. It uses Globus services for remote data management and funcX for scalable remote execution. Gladier provides tools and a client to define workflows that span multiple facilities and orchestrate the execution of actions like data transfers, analysis, and publication. Example deployments at facilities like the Advanced Photon Source automate workflows for experiments like XPCS and crystallography.
Introduction to Globus Compute for researchers.pdf
Introduction to Globus Compute for researchers.pdfSusanTussy1 We will provide an overview of the Globus Compute service and walk through the process for installing and configuring the Globus Compute single-user agent. This presentation is aimed at researchers who would like to run their code on remote machines. You will learn how to run Python code on any compute resource—from a laptop to a supercomputer—using a unified interface.
How HPC and large-scale data analytics are transforming experimental science
How HPC and large-scale data analytics are transforming experimental scienceinside-BigData.com NERSC serves as the primary high-performance computing and data facility for the U.S. Department of Energy, supporting extensive research in physical sciences across various domains including biology and energy. The document discusses the evolution of NERSC's systems, emphasizing upcoming capabilities, real-time data analysis, and the integration of machine learning in scientific research. It highlights the critical role of supercomputing in advancing experimental science by enabling simulations, data analysis, and innovative scientific discoveries.
Foundations for the Future of Science
Foundations for the Future of ScienceGlobus Foundations for the future of science discusses using artificial intelligence and machine learning to advance scientific research. Key points discussed include using AI to analyze large datasets, develop scientific models, and automate experimental workflows. The document also outlines several examples of how the Globus data platform is currently enabling AI-powered scientific applications across multiple domains. Overall, the document advocates that embracing "AI for science" has the potential to accelerate scientific discovery by overcoming limitations in human analysis capabilities and computational resources.
Enduring Impact in Data-Driven Science
Enduring Impact in Data-Driven ScienceGlobus This document summarizes a presentation about the enduring impact of data-driven science and the Globus platform. It discusses how data needs are increasing dramatically, with greater requirements for data integration across larger scientific teams. The Globus platform provides frictionless data collection, sharing, and analysis across multiple locations. It allows users to access data anywhere, transfer and share data, and catalog and search data. The platform also enables reliable automation of multi-step data workflows and allows users to build their own applications using Globus services.
A Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific DiscoveryIan Foster The document discusses the Globus platform, a research data service aimed at reducing data friction and enhancing scientific discovery through cloud-based solutions. It outlines the platform's architecture, services, and lessons learned, emphasizing its hybrid free/subscription model for sustainable operation. Globus has facilitated significant data transfers and user engagement within a wide range of academic and research institutions.
Global Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptxIan Foster The document discusses advancements in global science services, particularly through the Globus platform, which facilitates remote access to computing resources and data management for researchers worldwide. It highlights the integration of AI and various technologies, including cloud-hosted services and secure authentication, to enhance the efficiency of research workflows while addressing challenges like data friction and the need for reliable execution. The text emphasizes the importance of a continuum-aware computing model to facilitate global collaboration and innovation in scientific research.
What's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtraGlobus The document outlines recent advancements in Globus VAS, including a rebuilt technical foundation with enhanced security and compliance, a growing connected storage ecosystem, and new automation services for data management. Key features include a modern architecture for cloud storage integration, improved search capabilities for data discovery, and the introduction of managed services for automating tasks in research. The overall mission is to enhance the productivity of researchers in data-driven science through sustainable software solutions.
Rpi talk foster september 2011
Rpi talk foster september 2011Ian Foster The document discusses the challenges faced by small and medium-scale scientific projects in managing the increasing volume and complexity of data. It advocates for the adoption of Software as a Service (SaaS) to enhance research IT services, streamline data movement, and reduce costs through economies of scale. The use of cloud-based solutions, such as Globus Online, is proposed to improve efficiency in data management and facilitate collaboration among researchers.
Globus at the United States Geological Survey
Globus at the United States Geological SurveyGlobus The document outlines the reimagining of the USGS's science data repository utilizing the Globus platform to handle large datasets and enhance data accessibility. It describes the construction of a searchable Globus index from over 10,000 data releases and the deployment of a user-friendly portal for data discovery and management. Future steps include improving data release processes and implementing a new SDK for better integration with Globus mechanisms.
GlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening KeynoteGlobus Manage Data with Assurance
- Globus is a data transfer service that aims to increase the efficiency of researchers by enabling sustainable data sharing.
- It has over 138,000 registered users and has transferred over 600 petabytes of data between endpoints.
- The document discusses recent enhancements to Globus including new connectors, improved security features for regulated data, and plans to further develop the platform capabilities.
Cifar
CifarBill St. Arnaud This document discusses the growing issue of large quantities of data from scientific research and simulations. It notes that data volumes are doubling every 9 months for genomic sequencing and that various scientific projects are now generating petabytes and exabytes of data. Current solutions for data management and analysis are often ad-hoc and inadequate for small and medium scale research. The document advocates for a national research cyberinfrastructure strategy in Canada that could provide standardized services and leverage partnerships with international organizations and commercial cloud providers to support scientists in dealing with large and complex research data. It raises questions about leadership and roles for universities, research organizations, and government in developing solutions.
Big Process for Big Data @ PNNL, May 2013
Big Process for Big Data @ PNNL, May 2013Ian Foster This document discusses the development of data management services to help researchers more easily collect, move, share and analyze big data. It describes the Globus data management platform, which allows researchers to transfer large datasets between locations and share them with collaborators. The author discusses plans to expand Globus capabilities to support additional research workflows, and to develop these services sustainably through a software-as-a-service model. The goal is to automate and outsource common data tasks in a way that provides researchers with a seamless experience for managing their research data.
Accelerating Data-driven Discovery in Energy Science
Accelerating Data-driven Discovery in Energy ScienceIan Foster The document discusses the evolution of data-driven discovery in energy science, emphasizing the development of new tools and collaborative research methodologies to enhance data access and sharing. It highlights the importance of eliminating data friction through services like Globus, which streamline data transfer, analysis, and publication processes. Additionally, it outlines the need for discovery engines to facilitate the integration of simulation and experimental data in order to address complex scientific questions.
Accelerating Discovery via Science Services
Accelerating Discovery via Science ServicesIan Foster The document discusses the acceleration of scientific discovery through the use of automated data management services, outsourcing, and cloud-based platforms like Globus. It highlights the importance of eliminating data friction and creating discovery engines to enhance collaboration and efficiency in scientific research. Key themes include the automation of data publication, sharing, and analysis, as well as the need for services that address the entire research lifecycle.
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr The document discusses the need for a campus-scale high-performance cyberinfrastructure to support the increasing demands of data-intensive research in scientific fields. It highlights challenges with existing internet bandwidth and presents solutions, including dedicated optical channels and prototypes that facilitate data transmission, storage, and analysis. The document showcases collaborations across institutions and advancements in technology to enhance research capabilities in areas such as genomics and cosmology.
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryGlobus This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Globus and the Materials Data Facility are presented. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryIan Foster This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Argonne National Lab and the University of Chicago Globus project are provided. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
The Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, EvolutionIan Foster The Earth System Grid Federation (ESGF) is a distributed network of climate data servers that archives and shares model output data used by scientists worldwide. ESGF has led data archiving for the Coupled Model Intercomparison Project (CMIP) since its inception. The ESGF Holdings have grown significantly from CMIP5 to CMIP6 and are expected to continue growing rapidly. A new ESGF2 project funded by the US Department of Energy aims to modernize ESGF to handle exabyte scale data volumes through a new architecture based on centralized Globus services, improved data discovery tools, and data proximate computing capabilities.
ESnet6 and Smart Instruments
ESnet6 and Smart InstrumentsIan Foster ESnet6 provides an ultra-fast and reliable network that enables new smart instruments for 21st century science. The network capacity has increased dramatically over time, with 2022 bandwidth being 500,000 times greater than 1993. This network allows rapid data transfer between facilities, such as replicating 7 petabytes of climate data between three labs. It also enables fast assembly and use of new instruments like high energy diffraction microscopy that can perform an analysis in 31 seconds. The integrated research infrastructure provided by Globus further supports use of remote resources and smart instruments that will drive scientific discovery.
More Related Content
Similar to Linking Scientific Instruments and Computation (20)
Computation and Knowledge
Computation and KnowledgeIan Foster The document discusses how computation can accelerate the generation of new knowledge by enabling large-scale collaborative research and extracting insights from vast amounts of data. It provides examples from astronomy, physics simulations, and biomedical research where computation has allowed more data and researchers to be incorporated, advancing various fields more quickly over time. Computation allows for data sharing, analysis, and hypothesis generation at scales not previously possible.
Better Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumIan Foster This document discusses the computing continuum and efforts to enable better information faster through computation. It provides examples of how techniques like executing tasks closer to data sources or on specialized hardware can significantly accelerate applications. Programming models and managed services are explored for specifying and executing workloads across diverse infrastructure. There are still open questions around optimizing networks, algorithms, and applications for the computing continuum.
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...
Gladier: The Globus Architecture for Data Intensive Experimental Research (AP...Globus Gladier is a framework that combines instruments, storage, and compute using loosely coupled services to automate and simplify data-intensive experimental research workflows. It uses Globus services for remote data management and funcX for scalable remote execution. Gladier provides tools and a client to define workflows that span multiple facilities and orchestrate the execution of actions like data transfers, analysis, and publication. Example deployments at facilities like the Advanced Photon Source automate workflows for experiments like XPCS and crystallography.
Introduction to Globus Compute for researchers.pdf
Introduction to Globus Compute for researchers.pdfSusanTussy1 We will provide an overview of the Globus Compute service and walk through the process for installing and configuring the Globus Compute single-user agent. This presentation is aimed at researchers who would like to run their code on remote machines. You will learn how to run Python code on any compute resource—from a laptop to a supercomputer—using a unified interface.
How HPC and large-scale data analytics are transforming experimental science
How HPC and large-scale data analytics are transforming experimental scienceinside-BigData.com NERSC serves as the primary high-performance computing and data facility for the U.S. Department of Energy, supporting extensive research in physical sciences across various domains including biology and energy. The document discusses the evolution of NERSC's systems, emphasizing upcoming capabilities, real-time data analysis, and the integration of machine learning in scientific research. It highlights the critical role of supercomputing in advancing experimental science by enabling simulations, data analysis, and innovative scientific discoveries.
Foundations for the Future of Science
Foundations for the Future of ScienceGlobus Foundations for the future of science discusses using artificial intelligence and machine learning to advance scientific research. Key points discussed include using AI to analyze large datasets, develop scientific models, and automate experimental workflows. The document also outlines several examples of how the Globus data platform is currently enabling AI-powered scientific applications across multiple domains. Overall, the document advocates that embracing "AI for science" has the potential to accelerate scientific discovery by overcoming limitations in human analysis capabilities and computational resources.
Enduring Impact in Data-Driven Science
Enduring Impact in Data-Driven ScienceGlobus This document summarizes a presentation about the enduring impact of data-driven science and the Globus platform. It discusses how data needs are increasing dramatically, with greater requirements for data integration across larger scientific teams. The Globus platform provides frictionless data collection, sharing, and analysis across multiple locations. It allows users to access data anywhere, transfer and share data, and catalog and search data. The platform also enables reliable automation of multi-step data workflows and allows users to build their own applications using Globus services.
A Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific DiscoveryIan Foster The document discusses the Globus platform, a research data service aimed at reducing data friction and enhancing scientific discovery through cloud-based solutions. It outlines the platform's architecture, services, and lessons learned, emphasizing its hybrid free/subscription model for sustainable operation. Globus has facilitated significant data transfers and user engagement within a wide range of academic and research institutions.
Global Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptxIan Foster The document discusses advancements in global science services, particularly through the Globus platform, which facilitates remote access to computing resources and data management for researchers worldwide. It highlights the integration of AI and various technologies, including cloud-hosted services and secure authentication, to enhance the efficiency of research workflows while addressing challenges like data friction and the need for reliable execution. The text emphasizes the importance of a continuum-aware computing model to facilitate global collaboration and innovation in scientific research.
What's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtraGlobus The document outlines recent advancements in Globus VAS, including a rebuilt technical foundation with enhanced security and compliance, a growing connected storage ecosystem, and new automation services for data management. Key features include a modern architecture for cloud storage integration, improved search capabilities for data discovery, and the introduction of managed services for automating tasks in research. The overall mission is to enhance the productivity of researchers in data-driven science through sustainable software solutions.
Rpi talk foster september 2011
Rpi talk foster september 2011Ian Foster The document discusses the challenges faced by small and medium-scale scientific projects in managing the increasing volume and complexity of data. It advocates for the adoption of Software as a Service (SaaS) to enhance research IT services, streamline data movement, and reduce costs through economies of scale. The use of cloud-based solutions, such as Globus Online, is proposed to improve efficiency in data management and facilitate collaboration among researchers.
Globus at the United States Geological Survey
Globus at the United States Geological SurveyGlobus The document outlines the reimagining of the USGS's science data repository utilizing the Globus platform to handle large datasets and enhance data accessibility. It describes the construction of a searchable Globus index from over 10,000 data releases and the deployment of a user-friendly portal for data discovery and management. Future steps include improving data release processes and implementing a new SDK for better integration with Globus mechanisms.
GlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening KeynoteGlobus Manage Data with Assurance
- Globus is a data transfer service that aims to increase the efficiency of researchers by enabling sustainable data sharing.
- It has over 138,000 registered users and has transferred over 600 petabytes of data between endpoints.
- The document discusses recent enhancements to Globus including new connectors, improved security features for regulated data, and plans to further develop the platform capabilities.
Cifar
CifarBill St. Arnaud This document discusses the growing issue of large quantities of data from scientific research and simulations. It notes that data volumes are doubling every 9 months for genomic sequencing and that various scientific projects are now generating petabytes and exabytes of data. Current solutions for data management and analysis are often ad-hoc and inadequate for small and medium scale research. The document advocates for a national research cyberinfrastructure strategy in Canada that could provide standardized services and leverage partnerships with international organizations and commercial cloud providers to support scientists in dealing with large and complex research data. It raises questions about leadership and roles for universities, research organizations, and government in developing solutions.
Big Process for Big Data @ PNNL, May 2013
Big Process for Big Data @ PNNL, May 2013Ian Foster This document discusses the development of data management services to help researchers more easily collect, move, share and analyze big data. It describes the Globus data management platform, which allows researchers to transfer large datasets between locations and share them with collaborators. The author discusses plans to expand Globus capabilities to support additional research workflows, and to develop these services sustainably through a software-as-a-service model. The goal is to automate and outsource common data tasks in a way that provides researchers with a seamless experience for managing their research data.
Accelerating Data-driven Discovery in Energy Science
Accelerating Data-driven Discovery in Energy ScienceIan Foster The document discusses the evolution of data-driven discovery in energy science, emphasizing the development of new tools and collaborative research methodologies to enhance data access and sharing. It highlights the importance of eliminating data friction through services like Globus, which streamline data transfer, analysis, and publication processes. Additionally, it outlines the need for discovery engines to facilitate the integration of simulation and experimental data in order to address complex scientific questions.
Accelerating Discovery via Science Services
Accelerating Discovery via Science ServicesIan Foster The document discusses the acceleration of scientific discovery through the use of automated data management services, outsourcing, and cloud-based platforms like Globus. It highlights the importance of eliminating data friction and creating discovery engines to enhance collaboration and efficiency in scientific research. Key themes include the automation of data publication, sharing, and analysis, as well as the need for services that address the entire research lifecycle.
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr The document discusses the need for a campus-scale high-performance cyberinfrastructure to support the increasing demands of data-intensive research in scientific fields. It highlights challenges with existing internet bandwidth and presents solutions, including dedicated optical channels and prototypes that facilitate data transmission, storage, and analysis. The document showcases collaborations across institutions and advancements in technology to enhance research capabilities in areas such as genomics and cosmology.
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryGlobus This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Globus and the Materials Data Facility are presented. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryIan Foster This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Argonne National Lab and the University of Chicago Globus project are provided. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
More from Ian Foster (20)
The Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, EvolutionIan Foster The Earth System Grid Federation (ESGF) is a distributed network of climate data servers that archives and shares model output data used by scientists worldwide. ESGF has led data archiving for the Coupled Model Intercomparison Project (CMIP) since its inception. The ESGF Holdings have grown significantly from CMIP5 to CMIP6 and are expected to continue growing rapidly. A new ESGF2 project funded by the US Department of Energy aims to modernize ESGF to handle exabyte scale data volumes through a new architecture based on centralized Globus services, improved data discovery tools, and data proximate computing capabilities.
ESnet6 and Smart Instruments
ESnet6 and Smart InstrumentsIan Foster ESnet6 provides an ultra-fast and reliable network that enables new smart instruments for 21st century science. The network capacity has increased dramatically over time, with 2022 bandwidth being 500,000 times greater than 1993. This network allows rapid data transfer between facilities, such as replicating 7 petabytes of climate data between three labs. It also enables fast assembly and use of new instruments like high energy diffraction microscopy that can perform an analysis in 31 seconds. The integrated research infrastructure provided by Globus further supports use of remote resources and smart instruments that will drive scientific discovery.
Foster CRA March 2022.pptx
Foster CRA March 2022.pptxIan Foster The document discusses the concept of a national discovery cloud designed to enhance human intellect through advanced computing and data integration. It highlights the importance of creating new tools, platforms, and educational initiatives to facilitate research and innovation. Key challenges include ensuring privacy, balancing resource needs, and integrating diverse capabilities into a cohesive framework for the R&D community.
Big Data, Big Computing, AI, and Environmental Science
Big Data, Big Computing, AI, and Environmental ScienceIan Foster The document discusses three transformative technologies: exascale computing, big data, and artificial intelligence, which are significantly enhancing scientific research capabilities. It outlines the advancements in AI for science, particularly using deep learning for applications like automated cloud classification and climate modeling, while emphasizing the integration of large-scale data from satellites. Additionally, it describes the processing and classification of extensive MODIS satellite imagery to develop insights into cloud behavior and environmental phenomena.
AI at Scale for Materials and Chemistry
AI at Scale for Materials and ChemistryIan Foster The document discusses using artificial intelligence (AI) to accelerate materials innovation for clean energy applications. It outlines six elements needed for a Materials Acceleration Platform: 1) automated experimentation, 2) AI for materials discovery, 3) modular robotics for synthesis and characterization, 4) computational methods for inverse design, 5) bridging simulation length and time scales, and 6) data infrastructure. Examples of opportunities include using AI to bridge simulation scales, assist complex measurements, and enable automated materials design. The document argues that a cohesive infrastructure is needed to make effective use of AI, data, computation, and experiments for materials science.
Coding the Continuum
Coding the ContinuumIan Foster The document discusses the evolution of computing and networking, emphasizing the concept of the 'computing continuum' where computing resources are distributed across various types of appliances and locations. It highlights technological advancements, such as 5G and cloud computing, and presents models for optimizing performance based on speed, location, and resource utilization. The narrative ties together concepts of latency, computational efficiency, and cost-management in modern computing environments.
Data Tribology: Overcoming Data Friction with Cloud Automation
Data Tribology: Overcoming Data Friction with Cloud AutomationIan Foster The document discusses the concept of data tribology, which seeks to reduce 'data friction' through cloud automation and services. It highlights the various tasks that can be automated in research data handling, such as authentication, data transfer, sharing, and processing. The Globus platform is presented as a successful example of implementing these principles to enhance data accessibility and usability in research environments.
Scaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and JupyterIan Foster The document discusses the integration of Globus services with JupyterHub to enhance collaborative data science, enabling authentication, data access, movement, sharing, and search within Jupyter notebooks. It emphasizes the need for scalable, reproducible environments for data analysis through containers and incorporating parallel computing using the Parsl library. Additionally, it highlights features like remote data access, secure token management, and APIs for seamless interactions with distributed storage systems.
Learning Systems for Science
Learning Systems for ScienceIan Foster Deep learning is finding applications in science such as predicting material properties. DLHub is being developed to facilitate sharing of deep learning models, data, and code for science. It will collect, publish, serve, and enable retraining of models on new data. This will help address challenges of applying deep learning to science like accessing relevant resources and integrating models into workflows. The goal is to deliver deep learning capabilities to thousands of scientists through software for managing data, models and workflows.
Team Argon Summary
Team Argon SummaryIan Foster Team Argon proposes a commons platform using reusable components to promote continuous FAIRness of data. These components include Globus Connect Server for standardized data access and transfer across storage systems, Globus Auth for authentication and authorization, and BDBags for exchange of query results and cohorts using a common manifest format. Together these aim to provide uniform, secure, and reliable access, transfer, and sharing of data while supporting identification, search, and virtualization of derived data products.
Thoughts on interoperability
Thoughts on interoperabilityIan Foster This document discusses lessons learned for achieving interoperability. It recommends having a clear purpose, starting with basic conventions like identifiers, monitoring commitments to build trust, and focusing on outward-facing interoperability through simple APIs and platforms rather than full software stacks. Observance of industry practices like authentication methods and cloud-based platforms is also advised to promote rapid development and distribution of applications.
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...Ian Foster The document discusses advances in online data analysis and reduction for extreme scale computing, highlighting the integration of automation to enhance the research data lifecycle. Emphasizing the importance of real-time analysis, it outlines the challenges faced in handling large datasets and proposes a co-optimized approach that integrates simulation, analysis, and data reduction. It also introduces the Codar initiative, which aims to develop infrastructure for effective online data services and addresses key research questions related to data management and error impact.
NIH Data Commons Architecture Ideas
NIH Data Commons Architecture IdeasIan Foster The document outlines the implementation of a commons platform for promoting continuous fairness in data sharing, emphasizing the need for data to be findable, accessible, interoperable, and reusable. It highlights essential components like Globus Auth for secure authentication, Globus Connect Server for data transfer, and the use of standardized identifiers and formats for data management and sharing. The infrastructure aims to support scalability, interoperability, and user-friendly experiences for diverse collaboration and virtual cohort creation.
Going Smart and Deep on Materials at ALCF
Going Smart and Deep on Materials at ALCFIan Foster The document discusses the development of an interactive and scalable data service for advanced materials science at the Argonne Leadership Computing Facility (ALCF), emphasizing the integration of advanced data analytics and machine learning to enhance simulation capabilities. It outlines methodologies for making data more accessible, automating processes, and creating reusable data objects to promote reproducibility in scientific research. Additionally, it highlights the application of time-dependent density functional theory and machine learning in predicting stopping power in materials, illustrating the significance of this service in improving computational efficiency and scientific discovery.
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...Ian Foster This document discusses computing challenges posed by rapidly increasing data scales in scientific applications and high performance computing. It introduces the concept of online data analysis and reduction as an alternative to traditional offline analysis to help address these challenges. The key messages are that dramatic changes in HPC system geography due to different growth rates of technologies are driving new application structures and computational logistics problems, presenting exciting new computer science opportunities in online data analysis and reduction.
Software Infrastructure for a National Research Platform
Software Infrastructure for a National Research PlatformIan Foster Ian Foster's talk at the 1st National Research Platform Workshop focuses on the importance of software infrastructure for a national research platform, emphasizing accessibility, performance, security, reliability, programmability, manageability, and sustainability. Globus is presented as a solution that facilitates data transfer, sharing, and publication across various storage systems, benefiting researchers and institutions. The platform supports over 10,000 active endpoints and boasts a 99.5% uptime, showcasing its capability to streamline collaboration and data management in scientific research.
Accelerating the Experimental Feedback Loop: Data Streams and the Advanced Ph...
Accelerating the Experimental Feedback Loop: Data Streams and the Advanced Ph...Ian Foster The Advanced Photon Source (APS) at Argonne National Laboratory produces intense beams of x-rays for scientific research. Experimental data from the APS is growing dramatically due to improved detectors and a planned upgrade. This is creating data and computation challenges across the entire experimental process. Efforts are underway to accelerate the experimental feedback loop through automated data analysis, optimized data streaming, and computer-steered experiments to minimize data collection. The goal is to enable real-time insights and knowledge-driven experiments.
Materials Data Facility: Streamlined and automated data sharing, discovery, ...
Materials Data Facility: Streamlined and automated data sharing, discovery, ...Ian Foster The document details advancements in the Materials Data Facility (MDF) focused on streamlining and automating data sharing, discovery, access, and analysis for materials science. Key highlights include the ingestion of significant datasets, the deployment of APIs for community development, and the establishment of unified search across multiple data sources. Future steps involve enhancing data indexing and search capabilities while continuing to engage with the materials science community and educational institutions.
Globus Auth: A Research Identity and Access Management Platform
Globus Auth: A Research Identity and Access Management PlatformIan Foster Globus Auth is an identity and access management platform designed for researchers, enabling secure and reliable file sharing, authentication, and authorization across various services and applications. It simplifies the integration of user login and OAuth2 support, allowing users to access services without needing local accounts and leveraging existing identities. With a focus on interoperability and security, Globus Auth is positioned to empower the research community by creating a cohesive ecosystem of integrated services.
Streamlined data sharing and analysis to accelerate cancer research
Streamlined data sharing and analysis to accelerate cancer researchIan Foster The document discusses how Globus serves as a data-sharing platform that accelerates cancer research by streamlining scientific data sharing and analysis. It highlights the challenges in data management, the capabilities of Globus in enhancing secure data transfers, and its role in the identification of germline mutations related to blood cancers. The emphasis is on the ease of use, secure access, and automation features of Globus, which have made it widely adopted in research environments.
Ad
Recently uploaded (20)
AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?Shashi Sathyanarayana, Ph.D This slide illustrates a side-by-side comparison between human-written, AI-written, and ambiguous content. It highlights subtle cues that help readers assess authenticity, raising essential questions about the future of communication, trust, and thought leadership in the age of generative AI.
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdfNilesh Gule Slide deck related to the GitHub Copilot Bootcamp in Melbourne on 17 June 2025
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech EnthusiastRaman Bhaumik A Junior Software Developer with a flair for innovation, Raman Bhaumik excels in delivering scalable web solutions. With three years of experience and a solid foundation in Java, Python, JavaScript, and SQL, she has streamlined task tracking by 20% and improved application stability.
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo1pcity Studios, Inc AI VIDEO MAGAZINE - r/aivideo community newsletter – Exclusive Tutorials: How to make an AI VIDEO from scratch, PLUS: How to make AI MUSIC, Hottest ai videos of 2025, Exclusive Interviews, New Tools, Previews, and MORE - JUNE 2025 ISSUE -
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdfFwdays Design patterns like Event Sourcing and Event Streaming have long become standards for building real-time analytics systems. However, when the system load becomes nonlinear with fast and often unpredictable spikes, it's crucial to respond quickly in order not to lose real-time operating itself.
In this talk, I’ll share my experience implementing and using a tool like Temporal.io. We'll explore the evolution of our system for maintaining real-time report generation and discuss how we use Temporal both for short-lived pipelines and long-running background tasks.
cnc-processing-centers-centateq-p-110-en.pdf
cnc-processing-centers-centateq-p-110-en.pdfAmirStern2 מרכז עיבודים תעשייתי בעל 3/4/5 צירים, עד 22 החלפות כלים עם כל אפשרויות העיבוד הדרושות. בעל שטח עבודה גדול ומחשב נוח וקל להפעלה בשפה העברית/רוסית/אנגלית/ספרדית/ערבית ועוד..
מסוגל לבצע פעולות עיבוד שונות המתאימות לענפים שונים: קידוח אנכי, אופקי, ניסור, וכרסום אנכי.
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdfPriyanka Aash Cracking the Code - Unveiling Synergies Between Open Source Security and AI
Securing Account Lifecycles in the Age of Deepfakes.pptx
Securing Account Lifecycles in the Age of Deepfakes.pptxFIDO Alliance Securing Account Lifecycles in the Age of Deepfakes
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdfOpenMetadata The community meetup was held Wednesday June 18, 2025 @ 9:00 AM PST.
Catch the next OpenMetadata Community Meetup @ https://www.meetup.com/openmetadata-meetup-group/
In this month's OpenMetadata Community Meetup, "Enforcing Quality & SLAs with OpenMetadata Data Contracts," we covered data contracts, why they matter, and how to implement them in OpenMetadata to increase the quality of your data assets!
Agenda Highlights:
👋 Introducing Data Contracts: An agreement between data producers and consumers
📝 Data Contracts key components: Understanding a contract and its purpose
🧑🎨 Writing your first contract: How to create your own contracts in OpenMetadata
🦾 An OpenMetadata MCP Server update!
➕ And More!
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my billstijn40 VMware’s pricing changes have forced organizations to rethink their datacenter cost management strategies. While FinOps is commonly associated with cloud environments, the FinOps Foundation has recently expanded its framework to include Scopes—and Datacenter is now officially part of the equation. In this session, we’ll map the FinOps Framework to a VMware-based datacenter, focusing on cost visibility, optimization, and automation. You’ll learn how to track costs more effectively, rightsize workloads, optimize licensing, and drive efficiency—all without migrating to the cloud. We’ll also explore how to align IT teams, finance, and leadership around cost-aware decision-making for on-prem environments. If your VMware bill keeps increasing and you need a new approach to cost management, this session is for you!
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdfPriyanka Aash Lessons Learned from Developing Secure AI Workflows
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdfPriyanka Aash Techniques for Automatic Device Identification and Network Assignment
10 Key Challenges for AI within the EU Data Protection Framework.pdf
10 Key Challenges for AI within the EU Data Protection Framework.pdfPriyanka Aash 10 Key Challenges for AI within the EU Data Protection Framework
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes ProbableSaikat Basu Imagine combining the "brains" of Artificial Intelligence (AI) with the "super muscles" of Quantum Computing. That's Quantum AI!
It's a new field that uses the mind-bending rules of quantum physics to make AI even more powerful.
The Future of Product Management in AI ERA.pdf
The Future of Product Management in AI ERA.pdfAlyona Owens Hi, I’m Aly Owens, I have a special pleasure to stand here as over a decade ago I graduated from CityU as an international student with an MBA program. I enjoyed the diversity of the school, ability to work and study, the network that came with being here, and of course the price tag for students here has always been more affordable than most around.
Since then I have worked for major corporations like T-Mobile and Microsoft and many more, and I have founded a startup. I've also been teaching product management to ensure my students save time and money to get to the same level as me faster avoiding popular mistakes. Today as I’ve transitioned to teaching and focusing on the startup, I hear everybody being concerned about Ai stealing their jobs… We’ll talk about it shortly.
But before that, I want to take you back to 1997. One of my favorite movies is “Fifth Element”. It wowed me with futuristic predictions when I was a kid and I’m impressed by the number of these predictions that have already come true. Self-driving cars, video calls and smart TV, personalized ads and identity scanning. Sci-fi movies and books gave us many ideas and some are being implemented as we speak. But we often get ahead of ourselves:
Flying cars,Colonized planets, Human-like AI: not yet, Time travel, Mind-machine neural interfaces for everyone: Only in experimental stages (e.g. Neuralink).
Cyberpunk dystopias: Some vibes (neon signs + inequality + surveillance), but not total dystopia (thankfully).
On the bright side, we predict that the working hours should drop as Ai becomes our helper and there shouldn’t be a need to work 8 hours/day. Nobody knows for sure but we can require that from legislation. Instead of waiting to see what the government and billionaires come up with, I say we should design our own future.
So, we as humans, when we don’t know something - fear takes over. The same thing happened during the industrial revolution. In the Industrial Era, machines didn’t steal jobs—they transformed them but people were scared about their jobs. The AI era is making similar changes except it feels like robots will take the center stage instead of a human. First off, even when it comes to the hottest space in the military - drones, Ai does a fraction of work. AI algorithms enable real-time decision-making, obstacle avoidance, and mission optimization making drones far more autonomous and capable than traditional remote-controlled aircraft. Key technologies include computer vision for object detection, GPS-enhanced navigation, and neural networks for learning and adaptation. But guess what? There are only 2 companies right now that utilize Ai in drones to make autonomous decisions - Skydio and DJI.
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...Fwdays What can go wrong if you allow each service to access the database directly? In a startup, this seems like a quick and easy solution, but as the system scales, problems appear that no one could have guessed.
In my talk, I'll share Solidgate's experience in transforming its architecture: from the chaos of direct connections to a service-based data access model. I will talk about the transition stages, bottlenecks, and how isolation affected infrastructure support. I will honestly show what worked and what didn't. In short, we will analyze the controversy of this talk.
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to productionMichele Leroux Bustamante We live in an ever evolving landscape for cyber threats creating security risk for your production systems. Mitigating these risks requires participation throughout all stages from development through production delivery - and by every role including architects, developers QA and DevOps engineers, product owners and leadership. No one is excused! This session will cover examples of common mistakes or missed opportunities that can lead to vulnerabilities in production - and ways to do better throughout the development lifecycle.
You are not excused! How to avoid security blind spots on the way to production
You are not excused! How to avoid security blind spots on the way to productionMichele Leroux Bustamante
Ad
Linking Scientific Instruments and Computation
- 1. Linking Scientific Instruments and Computation: Patterns, Technologies, Experiences Ian Foster The University of Chicago Argonne National Laboratory [email protected] Crescat scientia; vita excolatur https://arxiv.org/abs/2204.05128 https://arxiv.org/abs/2208.09513
- 2. A new generation of scientific instruments New sensors produce data at high velocities and in large volumes New methods and structures are required to capture and process data, and to feed back to sensors Increasing need to harness HPC, cloud, edge computers An instrument becomes a set of flows, overlaid on distributed physical resources and software Mark Boland, https://bit.ly/3cfSosk, 2017
- 3. Example: High-energy diffraction microscopy
- 5. Example: Serial synchrotron crystallography
- 6. A modular, extensible approach to creating and running flows Flows Capture useful patterns as sequences of actions. Resource-independent
- 7. A modular, extensible approach to creating and running flows Flows Capture useful patterns as sequences of actions. Resource-independent Action providers Implement actions. Resource-independent Compute Action Provider: Run function at A. Transfer Action Provider: Transfer from A to B. Search Action Provider: Publish metadata. …
- 8. A modular, extensible approach to creating and running flows Flows Capture useful patterns as sequences of actions. Resource-independent Action providers Implement actions. Resource-independent Fabric Implements auth, data, and compute APIs for manipulating resources Authenticate user. Delegate credentials. Manage file transfers. Run jobs on computers. Access data catalog. … Compute Action Provider: Run function at A. Transfer Action Provider: Transfer from A to B. Search Action Provider: Publish metadata. …
- 9. Builds on cloud-hosted Globus automation services Globus automation services Triggers Flows Analysis Computer Timers Queues Step Step Step Step Event Type: creation Match: *tiff Action Queue 1 2 3 4 Action Type: user selection data: <feature extraction> Options: approve/reject Microscope Step Step Step Step Flow run Step Step Step Step Action Type: transfer From: microscope To: analysis computer https://arxiv.org/abs/2204.05128
- 10. Capture flows in reusable forms In various ways: - YAML documents - Python “Gladier” SDK - Web authoring
- 11. Customize flow to application Capture flows in reusable forms
- 12. Customize flow to application Specialize flow to resources Capture flows in reusable forms
- 13. Check flow status Execute specialized flow Customize flow to application Specialize flow to resources Capture flows in reusable forms
- 14. Execute specialized flow Customize flow to application Specialize flow to resources Capture flows in reusable forms Examine flow actions
- 15. Execute specialized flow Customize flow to application Specialize flow to resources Capture flows in reusable forms Identify failed actions
- 16. AI model training AI model deployment Data collection & transfer Cerebras Catalog & publish Detector Injector x-ray Target FAIR data Data reduction, refine structures Data collection & transfer AI accelerators, HPC Ptychographic reconstruction Data collection & transfer (raw) Data collection & transfer (position) AI accelerators Serial synchrotron crystallography Ptychography High energy diffraction microscopy Flows have been developed for light source data analysis, biomedical and materials science data ingest, on-demand simulation, …
- 17. Determining protein structures 10-100x faster “These data services have taken the time to solve a structure from weeks to days and now to hours” Darren Sherrell, SBC beamline scientist APS Sector 19 • Developed new automation pipeline to collect data, analyze and visualize the data, solve protein structure and load results into a searchable portal for real-time feedback • Achieved over 10-100x speed up in time to solution of protein structures at APS beamline • Leveraged unique DOE facilities at Advanced Photon Source (SBC Sector 19) and ALCF (Theta/ ThetaGPU, Petrel, and Data Portals) Deposited first results in open repositories Automation pipeline (Chard, Vescovi, Foster, Blaiszik, Sherrell, Joachimiak, et al.) ALCF Theta ALCF Theta ALCF Theta Data Portals APS ALCF Petrel ALCF Theta 17
- 18. Flow invocations 2020-21 for five APS experiments Numbers vary due to facility and experimental schedules.
- 19. We collect detailed performance data on flows https://arxiv.org/abs/2204.05128 Transfer, compute, and cataloging costs for median flows
- 20. Round-trip latencies for various action providers • Current architecture has ~1 sec minimum latency due to cloud interaction • funcX latencies higher due to polling strategy • Both can be improved as needed
- 21. We build on a universal auth, compute, & data fabric Globus Auth Authentication and delegation mechanisms to control what happens where Run functions anywhere funcX deployed Access data anywhere Globus Connect deployed * See also: Integrated Research Infrastructure, computing continuum, grid Globus Connect
- 22. As of 4/2022
- 23. Globus hybrid “SaaS” model: Data fabric
- 24. Globus hybrid “SaaS” model: Compute fabric funcX agent funcX agent Customer owned and administered computer with funcX agent running on it funcX service orchestrates function execution via communication with funcX agent
- 25. Polaris Bebop Cluster Argonne Leadership Computing Facility Laboratory Computing Research Center Eagle store APS Computing Orthros Cluster APS DM system Portal server Portal server Theta Advanced Photon Source Key: funcX agent Globus Connect agent API API API User-defined flows Globus-accessible storage and computing (10,000s of systems) Globus Automation Services
- 26. Building computationally-enhanced instruments: There is much more to be done! • We have worked so far with light sources and data ingest pipelines • We are pleased with adaptability and reliability • Work required in capability (e.g., iteration) and performance • Others are applying tools to microscopes and other instruments • New action providers are needed for instrument control • We are eager to find partners who want to work with us on developing and/or applying these methods and tools!
- 27. Thanks to talented colleagues! Linking Scientific Instruments & HPC: Patterns, Technologies, Experiences Globus Automation Services: Research process automation across the space-time continuum Rachana Ananthakrishnan Josh Bryan Kyle Chard Ryan Chard Kurt McKee Jim Pruyne Brigitte Raumann https://arxiv.org/abs/2204.05128 https://arxiv.org/abs/2208.09513 Raf Vescovi Ryan Chard Nick Saint Ben Blaiszik Jim Pruyne Tekin Bicer Alex Lavens Zhengchun Liu Mike Papka Suresh Narayanan Nicholas Schwarz Kyle Chard and And sponsors And the rest of the ALCF, APS, & Globus teams
- 28. Recap: Enabling new instruments Reusable flows composed from an extensible set of actions Built on global auth, compute, data fabric Join us in applying these methods! https://arxiv.org/abs/2204.05128 https://arxiv.org/abs/2208.09513 https://www.globus.org/platform/services/flows
Editor's Notes
- #3: Probe. Instrument. Meter.
- #4: Metacomputing revisited 1010 x faster 105 x more tasks 106 x more data Link HPC, AI, instruments c still 3 x 108 m/s
- #5: Metacomputing revisited 1010 x faster 105 x more tasks 106 x more data Link HPC, AI, instruments c still 3 x 108 m/s
- #6: Metacomputing revisited 1010 x faster 105 x more tasks 106 x more data Link HPC, AI, instruments c still 3 x 108 m/s
- #28: Need to mention other Braid people! Eliu Huerta Bogdan Nicolae Justin Wozniak MENTION Eliu work?