2010 vampir workshop_iu_configuration

Apr 26, 20100 likes410 views

This document provides instructions for using the Vampir toolchain at Indiana University (IU) on the Quarry and BigRed clusters. It describes how to run Vampir, VampirServer, and VampirTrace on these clusters, including which software modules to load, how to run jobs in PBS, and where to find trace files. The document also lists the software versions of Vampir, VampirServer, VampirTrace, OpenMPI, and compilers available on each cluster.

How To Use The
Vampir Toolchain At IU

Robert Henschel
henschel@indiana.edu

April 2010

Contents
• Quarry
• BigRed

Robert Henschel

General
• Trace creation and visualization does not have to
happen on the same platform!
– You can look at traces created on Quarry (or your
own desktop) on BigRed

Robert Henschel

Quarry – SoftEnv Keys
• Vampir Version 5.2
+vampir-5.2-32 or +vampir-5.2-64
• VampirServer Version 2
+vampirserver-2-openmpi-64
• VampirTrace Version 5.8
@intel-11.1.059-64
+openmpi-1.4.1-intel-11.1.059-64
+vampirtrace-5.8.0-intel-openmpi-64
or
@intel-11.1.059-32
+openmpi-1.4.1-intel-11.1.059-32
+vampirtrace-5.8.0-intel-openmpi-32

Robert Henschel

Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
– On interactive nodes b005-b008

Robert Henschel

Quarry – Running Vampir
• Vampir Version 5.2
– Interactive job in PBS
qsub -I -X
soft add +vampir-5.2-64
vampir

– On interactive nodes b005-b008
• You share the node with other people, this is
especially a problem for opening large trace files!
ssh -X b005
soft add +vampir-5.2-64
vampir

Robert Henschel

Quarry – Running VampirServer
• Vampir Server Version 2
– Server in PBS, viewer on interactive node
– Server and viewer in interactive job in PBS
– Server and viewer on interactive nodes b005-b008

Robert Henschel

Quarry – Running VampirServer
• Vampir Server Version 2
– Server in PBS, viewer on interactive node
– Make sure your “.soft” file contains
@intel-11.1.059-64
+openmpi-1.4.1-intel-11.1.059-64
+vampirserver-2-openmpi-64
– Logout/Login to active changes in “.soft”
– PBS script
#PBS -l nodes=2:ppn=8,walltime=5:00:00
#PBS -m ae
#PBS -N VampirServer
#PBS
vampirserver --mode=mpi-mode --nproc=16 1>$HOME/vampir.out 2>&1
Start job
qsub vampir.pbs
– Wait for the job to start
Robert Henschel

Quarry – Running VampirServer
• Lookup host and port information
» It may take a minute or two after the job
started for this file to get updated!!!
cat $HOME/vampir.out
Running...
Server listens on: b052:30000

• Viewer on interactive nodes b005-b008
ssh -X b005
soft add +openmpi-1.4.1-intel-11.1.059-64
soft add +vampirserver-2-openmpi-64
vampir //b052:30000/path/to/*.otf

Robert Henschel

Quarry – Running VampirServer
• Interactive job in PBS
• Only one node
qsub -I -X
soft add +openmpi-1.4.1-intel-11.1.059-64
soft add +vampirserver-2-openmpi-64
vampirserver -m mpi-mode -n 8 1>output
2>&1 &
cat output
Running...
Server listens on: b048:30000

vampir //b048:30000/path/to/*.otf

Robert Henschel

Quarry – Running VampirServer
On interactive nodes b005-b008
• Only one node
• Runtime needs to be less than 24 hours
• You share the node with other people
ssh -X b005
soft add +openmpi-1.4.1-intel-11.1.059-64
soft add +vampirserver-2-openmpi-64
vampirserver -m mpi-mode -n 4 1>output
2>&1 &
cat output
Running...
Server listens on: b005:30000

vampir //b005:30000/path/to/*.otf

Robert Henschel

Quarry – Tracing with VampirTrace 5.8
• VampirTrace is configured to work with the Intel
compilers version 11.1 and OpenMPI 1.4.1

• Include the following keys in your .soft file
@intel-11.1.059-64
+openmpi-1.4.1-intel-11.1.059-64
+vampirtrace-5.8.0-intel-openmpi-64

• The VampirTrace compiler wrappers are now available
• It is possible to use them also with GCC, if you need to
do that, please send a mail to hpahelp@indiana.edu
• We can also provide a version for the PGI compilers.

Robert Henschel

BigRed – SoftEnv Keys
• Vampir Version 5.1
+vampir-5.1-64

• VampirServer Version 1.10
+vampirserver-1.10-64

• VampirTrace Version 5.6.3
+openmpi-1.3.1-ibm-64
+vampirtrace-5.6.3-ibm-64
or
+openmpi-1.3.1-ibm-32
+vampirtrace-5.6.3-ibm-32

Robert Henschel

In order to scale AdGear's real-time bidding (RTB) platform, we've been optimizing system and application performance. In this talk, I'll share all my findings, from basic tips to advanced topics. I'll cover coding patterns, metric collection, tracing and much more! Talk objectives: - Share basic tips, common pitfalls, efficient coding patterns - Introduce tooling for metric collection (statsderl, vmstats, system-stats, riak_sysmon) - Highlight erlang trace (fprof, flame graphs) - Expose advanced topics (VM tuning, lock-counter, systemtap, cpuset) Target audience: - Anyone looking to improve the performance of their Erlang application.

[네이버오픈소스세미나] Maglev Hashing Scheduler in IPVS, Linux Kernel - 송인주NAVER Engineering

This document discusses load balancing problems in container clusters and proposes using the IPVS Maglev hashing scheduler to provide an efficient and reliable load balancer. It describes how traditional load balancing approaches like static routing and ECMP can cause issues like disruption when containers are added or removed. The Maglev hashing algorithm is presented as a solution to provide consistent hashing for high availability. The document outlines implementing Maglev hashing in the IPVS module of the Linux kernel to leverage netfilter for packet processing and forwarding while avoiding issues of traditional load balancers. This achieves an efficient and reliable load balancer that can hash consistently without connection loss, even if load balancers fail.

systemd @ Facebook -- a year laterDavide Cavalca

This talk is a followup to Deploying systemd at scale that was presented at systemd.conf 2016, and covers the aftermath of the migration of our fleet to CentOS 7. Now that systemd is available everywhere, we found more and more services that started adopting it for their deployment, leveraging its features and occasionally exposing interesting behaviors. At the same time, we've been able to hone our process for integrating and rolling out new versions of systemd on the fleet, and started building tooling to manage and monitor it at scale.

Troubleshooting common oslo.messaging and RabbitMQ issuesMichael Klishin

This document discusses common issues with oslo.messaging and RabbitMQ and how to diagnose and resolve them. It provides an overview of oslo.messaging and how it uses RabbitMQ for RPC calls and notifications. Examples are given of where timeouts could occur in RPC calls. Methods for debugging include enabling debug logging, examining RabbitMQ queues and connections, and correlating logs from services. Specific issues covered include RAM usage, unresponsive nodes, rejected TCP connections, TLS connection failures, and high latency. General tips emphasized are using tools to gather data and consulting log files.

XenTT: Deterministic Systems Analysis in XenThe Linux Foundation

The document describes XenTT, a tool for deterministic replay in the Xen virtualization platform. XenTT records the execution history of a guest VM, including nondeterministic events like interrupts and I/O. It then replays the execution deterministically to allow for repeatable systems analysis. This is done by making the CPU and execution environment deterministic during replay and precisely recording the timing and location of nondeterministic events during the original run.

PuppetConf 2016: High Availability for Puppet – Russ Mull & Zack Smith, PuppetPuppet

ViFX Tech Connect: Containers & VMsViFX

This document discusses Docker container configuration and dependencies. It outlines how Docker containers depend on the Docker host for resources like CPU and memory allocation. It also describes mounting a remote NFS server volume to each Docker host to provide persistent storage for container data across all hosts. Commands are provided to stop the Docker service, mount the remote NFS volume, and run the Nginx container with port and CPU configuration options.

Fluentd v0.14 OverviewN Masahiro

The document summarizes the key features and changes in Fluentd v0.14, including new plugin APIs, plugin storage and helpers, time with nanosecond resolution, a ServerEngine-based supervisor for Windows support, and plans for symmetric multi-core processing, a counter API, and TLS/authentication in future versions. It also benchmarks some performance improvements and outlines the roadmap for Treasure Agent 3.0 based on Fluentd v0.14.

Running CentOS on the Facebook fleetDavide Cavalca

This talk will focus on how we deploy and manage CentOS on our fleet at Facebook, and showcase challenges, best practices and lessons learned working with a deployment of hundreds of thousands of machines. We'll discuss challenges encountered over the years, tools that we developed to overcome them, the process used to integrate upstream updates, packaging tools and workflows and configuration management challenges. The talk is mostly focused on bare metal, but will cover some container best practices as well. We'll also focus on our interactions with the RPM, Yum, Anaconda and systemd projects to showcase how to work with the upstream community.

Salt at schoolFlavio Castelli

This document discusses the deployment of Salt configuration management software across two school computer labs to standardize systems and reduce maintenance costs. Salt was used to automate the installation of Ubuntu, additional educational software, and configurations across over 45 computers with limited IT staff. Key benefits included easy replication of changes, enforcing a consistent state, and reducing hardware replacement needs. The architecture and modules of Salt allowed for flexible configuration and extension through Python scripts.

Packet Walk(s) In KubernetesDon Jayakody

When it comes to networking inside Kubernetes, selecting the correct networking solution may be one of the most important decisions you may face. This is especially true if you are trying to run a Kubernetes cluster in production. Therefore it's beneficial to have a good understanding of different CNI options out there and most importantly how these networking options are different from each other. This presentation goes over packet by packet-level details of how the network plumbing is happening with different CNI plugins including, Flannel, Calico & Cilium.

Your Inner Sysadmin - MidwestPHP 2015Chris Tankersley

One thing that most programmers do not take the time to understand is the servers that their application lives on. Most know a smattering of Apache configs, PHP configs, and basic information about the OS. This talk will deal with looking at tools that can help you quickly set up a server and how it can help you be a better developer. We'll look at tools like Puppet for server management, OSSEC for log management, different command line tools, and Nagios/Monit for system monitoring.

Epoll - from the kernel sidellj098

The document discusses the basics of I/O and event-driven I/O models in Linux like blocking I/O, non-blocking I/O, I/O multiplexing using select/poll and their internals. It then introduces epoll as a more efficient alternative to select/poll and describes its user-space API, kernel structures and how it works by adding file descriptors to an epoll instance and waking up blocked processes on I/O events.

OVS-NFV TutorialOpen Networking Perú (Opennetsoft)

1) The document provides an agenda and instructions for a hands-on tutorial on OVS/NFV basics using Open vSwitch, Linux containers, Docker, and virtual private networks. 2) It describes how to access two provided virtual machines and configure port mirroring with Open vSwitch to monitor network traffic between VMs. 3) Instructions are given for installing Linux containers on the VMs, configuring network interfaces and scripts, and testing connectivity between containers using GRE tunnels. 4) The tutorial also covers installing and configuring Docker containers on the VMs, creating virtual networks between them using GRE tunnels, and deploying example containers from Docker Hub.

Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyPuppet

This document summarizes lessons learned from deploying Puppet code globally at high speed. The key changes were moving from SVN to Git for version control, parallelizing deployments using MCollective instead of SSH loops, using MCollective policies instead of sudo, and switching to a pull model over push. These changes allowed deployments to be reduced from 4 minutes to 4 seconds. Environments were used to separate code for different teams and stages. A custom MCollective agent was created to deploy Git branches as Puppet environments. Cron jobs were used to pull updates to environments. Overall this approach improved the speed, consistency, and security of global Puppet deployments.

Deploying puppet code at light speedTomas Doran

Fluentd and PHPchobi e

Chobi_e is the author of the upcoming PECL Sundown package and contributor to other PHP projects involving logging and messaging. He discusses options for collecting log files in PHP applications, including using log file collectors like Fluentd which provide a robust and distributed logging solution. Chobi_e also introduces some Fluentd plugins he has created and potential other plugin ideas, and discusses setting up Fluentd logging in PHP applications.

zebra & openconfigd IntroductionKentaro Ebisawa

ReplicationPerforce

Perforce server replication allows a replica server to asynchronously mirror the data and transactions from a master server. It provides high availability, disaster recovery, and enables offloading of read-only workloads. The replication is performed using Perforce's journaling system, with the replica pulling missing journal entries and file revisions from the master server in the background. This allows setting up read-only replica servers with no external scripts required.

Perl Dist::Surveyor 2011Tim Bunce

This document describes Dist::Surveyor, a tool created by Tim Bunce to determine which CPAN distributions and versions are installed on a system by analyzing the contents of a local Perl library directory. It discusses the challenges involved and different approaches considered. The key features of Dist::Surveyor are that it uses MetaCPAN to match installed module files to candidate distributions, scores distributions based on included modules, and can generate a list of exact distributions to reinstall a matching library.

Twisted Introductioncyli

Phoenix at scaleSachirou Inoue

Your Inner Sysadmin - Tutorial (SunshinePHP 2015)Chris Tankersley

One thing that most programmers do not take the time to understand is the servers that their application lives on. Most know a smattering of Apache configs, PHP configs, and basic information about the OS. This talk will deal with looking at tools that can help you quickly set up a server and how it can help you be a better developer. We'll look at tools like puppet for server management, OSSEC for log management, different command line tools, and nagios/monit for system monitoring.

Linux Knowledge TransferTapio Vaattanen

The document discusses the history and development of Linux and Unix operating systems. It describes how Unix was initially developed at Bell Labs in 1969 and later licensed to other companies. It also outlines the creation of the GNU project in 1984 by Richard Stallman to develop a free and open source Unix-like operating system. The Linux kernel was later created by Linus Torvalds in 1991, combining the most popular GNU tools and utilities with the kernel to create a fully functional operating system, known as GNU/Linux. The document provides brief descriptions of important open source projects and utilities commonly found in Linux distributions.

Web scale infrastructures with kubernetes and flannelpurpleocean

La capacità di rispondere in poche frazioni di secondo alle richieste degli utenti - indipendentemente dal loro numero - è un fattore determinante per il successo dei servizi sul web. Secondo Amazon, bastano 100 millisecondi di latenza nella risposta per generare una perdita economica di circa l'1% sul fatturato [1]. In base alle statistiche di Google AdWords, inoltre, il 2015 ha sancito l’ufficiale superamento del numero di interazioni mobile rispetto a quelle desktop [2], con la conseguente riduzione della durata media delle sessioni di navigazione web. In uno scenario di questo tipo, la razionalizzazione dell’utilizzo delle risorse hardware e la capacità di scalare rispetto al numero di utenti sono fattori determinanti per il successo del business. In questo talk racconteremo la nostra esperienza di migrazione di soluzioni e-commerce di tipo enterprise in Magento da un’architettura basata su VM tradizionali ad una di tipo software-defined basata su Kubernetes, Flannel e Docker. Discuteremo, quindi, delle reali difficoltà da noi incontrate nel porting su container di soluzioni in produzione e daremo evidenza di come, alla fine di questo lungo viaggio, i nostri sforzi siano stati concretamente premiati dall’aumento di resilienza, affidabilità e automazione della soluzione finale. A supporto della conversazione, mostreremo i risultati dei benchmark da noi condotti per valutare la scalabilità della nuova architettura presentando delle evidenze delle reali capacità di Kubernetes come strumento di orchestrazione di servizi erogati in Docker container. Concluderemo l’intervento presentando il nostro progetto di distribuzione geografica dei nodi master di Kubernetes facendo uso di reti SD-WAN per garantire performance e continuità di servizio della soluzione.

Penetration Testing Resource Guide Bishop Fox

ITP Spacebrew Workshop - Spring 2014Brett Renfer

This document summarizes the schedule and content for a two-day Spacebrew workshop being held at NYU ITP and led by Brett Renfer. Day one will include introductions, an overview of Spacebrew basics and setup, exercises using Spacebrew libraries in Processing and JavaScript, and an assignment. Day two will include demos of assignments, documentation, and questions. Spacebrew is a service and toolkit that makes it easy to connect interactive things by routing data between applications using a client-server model over websockets. Custom data types and running one's own server are also discussed.

Pushing a camel through the eye of a needleSensePost

More Related Content

What's hot (20)

ViFX Tech Connect: Containers & VMsViFX

Fluentd v0.14 OverviewN Masahiro

Running CentOS on the Facebook fleetDavide Cavalca

Salt at schoolFlavio Castelli

Packet Walk(s) In KubernetesDon Jayakody

Your Inner Sysadmin - MidwestPHP 2015Chris Tankersley

Epoll - from the kernel sidellj098

OVS-NFV TutorialOpen Networking Perú (Opennetsoft)

Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyPuppet

Deploying puppet code at light speedTomas Doran

Fluentd and PHPchobi e

zebra & openconfigd IntroductionKentaro Ebisawa

ReplicationPerforce

Perl Dist::Surveyor 2011Tim Bunce

Twisted Introductioncyli

Phoenix at scaleSachirou Inoue

Your Inner Sysadmin - Tutorial (SunshinePHP 2015)Chris Tankersley

One thing that most programmers do not take the time to understand is the servers that their application lives on. Most know a smattering of Apache configs, PHP configs, and basic information about the OS. This talk will deal with looking at tools that can help you quickly set up a server and how it can help you be a better developer. We'll look at tools like puppet for server management, OSSEC for log management, different command line tools, and nagios/monit for system monitoring.

Linux Knowledge TransferTapio Vaattanen

Web scale infrastructures with kubernetes and flannelpurpleocean

Penetration Testing Resource Guide Bishop Fox

ViFX Tech Connect: Containers & VMsViFX

Fluentd v0.14 OverviewN Masahiro

Running CentOS on the Facebook fleetDavide Cavalca

Salt at schoolFlavio Castelli

Packet Walk(s) In KubernetesDon Jayakody

Your Inner Sysadmin - MidwestPHP 2015Chris Tankersley

Epoll - from the kernel sidellj098

OVS-NFV TutorialOpen Networking Perú (Opennetsoft)

Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyPuppet

Deploying puppet code at light speedTomas Doran

Fluentd and PHPchobi e

zebra & openconfigd IntroductionKentaro Ebisawa

ReplicationPerforce

Perl Dist::Surveyor 2011Tim Bunce

Twisted Introductioncyli

Phoenix at scaleSachirou Inoue

Your Inner Sysadmin - Tutorial (SunshinePHP 2015)Chris Tankersley

Linux Knowledge TransferTapio Vaattanen

Web scale infrastructures with kubernetes and flannelpurpleocean

Penetration Testing Resource Guide Bishop Fox

Similar to 2010 vampir workshop_iu_configuration (20)

ITP Spacebrew Workshop - Spring 2014Brett Renfer

Pushing a camel through the eye of a needleSensePost

RabbitMQ with python and ruby RuPy 2009Paolo Negri

Nodeconf npm 2011Florent Jaby ヅ

The document provides an overview of the Node Package Manager (npm). It discusses how npm works to reduce friction in the software development process by making it easy for developers to install packages and dependencies without conflicts. It describes npm's vision of avoiding "dependency hell" and its strategies for achieving this like ensuring consistent interfaces and reducing excessive metadata requirements. The document also summarizes key npm commands, how installations work, and future plans like binary distributions, an automated testing system called npat, and build farms to test packages on multiple platforms.

OpenNebulaConf 2014 - Lightning talk: OpenNebula Puppet Module - Norman Messt...OpenNebula Project

The document discusses the development of an OpenNebula Puppet Module to help manage and deploy over 980 VMs across 14 teams in a standardized way using Puppet, Hiera and RPM. It previously used a different virtualization tool that was not very DevOps friendly. The module handles database configuration, LDAP authentication, enabling or disabling Passenger, backups, and hooks. It also manages OpenNebula resources like hosts, clusters, datastores, images, vnets, context files and VMs using CLI tools. Future plans include switching to XML-RPC, supporting newer OpenNebula versions, performing upgrades, and integrating firewalls, load balancers and storage.

OpenNebula Conf 2014 | Lightning talk: OpenNebula Puppet Module - Norman Mess...NETWAYS

The document discusses the development of an OpenNebula Puppet Module to help manage and deploy virtual machines at Deutsche Post E-Post. It describes how the company previously lacked a suitable Puppet module for their virtualization tool, OpenNebula, and had an inefficient process with different tools used for development and production environments across 14 teams managing around 980 VMs. The new OpenNebula Puppet module addresses basic OpenNebula configuration, resources, and operations to help standardize and automate the company's virtual infrastructure deployment and management using Puppet, Hiera and OpenNebula.

Spacebrew @ SFPCBrett Renfer

- Spacebrew is a service and toolkit that makes it easy to connect interactive things like devices and apps over networks. - It uses a client-server model where clients can publish and subscribe to different data types. The server routes this data between clients. - Spacebrew supports different programming languages and platforms and aims to be implementation agnostic to connect different tools and languages for interactive projects.

Automation with ansibleKhizer Naeem

Alvaro Videla, Building a Distributed Data Ingestion System with RabbitMQTanya Denisyuk

This document contains a summary of a presentation about using RabbitMQ for messaging. It discusses RabbitMQ's features like multi-protocol support, polyglot clients, and large users. It then describes challenges like distributing an application across servers. RabbitMQ solutions to these include federation to replicate across servers, sharding queues for scaling, and federating queues for load balancing consumers. The presentation argues RabbitMQ provides tools to ingest, distribute, and scale messaging applications globally.

Encrypt your volumes with barbican open stack 2018Duncan Wannamaker

This document discusses encrypting volumes with Barbican in OpenStack. It provides an introduction to Barbican and encryption, describes how Barbican works with Nova and Cinder to provide transparent encryption of volumes using LUKS, discusses some issues the presenter encountered integrating Barbican on OnRamp's private cloud and how they were resolved, and limitations to be aware of when using encrypted volumes. It concludes with a demo and Q&A.

Построение распределенной системы сбора данных с помощью RabbitMQ, Alvaro Vid...Ontico

This document discusses building a distributed data ingestion system using RabbitMQ. It introduces RabbitMQ as a multi-protocol, polyglot messaging broker. The document then outlines some issues with a naïve ad-hoc solution to distributing data and proposes using RabbitMQ federation to address these issues. It provides an overview of how RabbitMQ federation works and how to configure it. Finally, it discusses additional RabbitMQ features like sharded queues and federated queues that can help scale the system.

Firebird on LinuxMind The Firebird

IPv6 for Pentesterscamsec

IPv6 for PentestersNotSoSecure Global Services

Who Broke My CryptoJohn Varghese

Usually we launch hundreds of instances in AWS for day to day work. As long as they are accessible from our hosts (probably a RHEL or Ubuntu or your own mac), we are good to go. But there are some instances where you might get a patch from IT for your host. Once you apply the patch, you realize that you are unable to access your AWS instances anymore. And your IT team doesn't have any clue on what happened. You contact AWS support, and they say it all looks good. So how do you proceed from this scenario? Where to start and what to do. This talk goes through all the steps starting with most basic checks all the way to updating the crypto key exchange algorithms on your host.

Scaling Servers and Storage for Film Assets Perforce

Pixar has scaled their use of Perforce over time to support increasing numbers of users, files, and data types associated with their animated films. They now have over 90 Perforce servers storing over 20 TB of data. To manage this scale, they utilize techniques like virtualization for flexible server provisioning, de-duplication to reduce storage usage, and scripts like Superp4 to automate management of metadata tables across multiple servers. While scaling has provided benefits, it has also introduced challenges around monitoring, performance, and administration across many interconnected systems.

Cooking a rabbit pieTomas Doran

This document discusses using AMQP and RabbitMQ for real-time system performance monitoring and job queueing. It describes key AMQP concepts like exchanges, queues, routing keys, and delivery modes. It then discusses how the speaker built a job queueing system called CatalystX::JobServer that uses these concepts and RabbitMQ. Demo examples are shown of using this system to queue and monitor the status of jobs. Potential next steps and other solutions like Gearman are also mentioned.

Bh fed-03-kaminskyDan Kaminsky

Using techniques like ARP spoofing and NAT, it is possible to acquire an IP address and internet access on a network without a DHCP server. By intercepting traffic between an existing node and gateway, one can insert themselves as the "man in the middle" and route traffic through a NAT configuration using the hijacked node's IP address. This allows acquiring internet access without a free IP address by multiplexing sessions through the NAT. Scanrand port scanning observations can also reveal network topology details like firewall locations through analysis of TTL values.

Designing a Highly Available Environment Using Methods of Modern IT Infrastru...Perforce

Hear about the multi-server Perforce architecture used at Remedy Entertainment, a developer of state-of-the-art action games, game franchises and cutting edge technology. Get tips on how various virtualization, storage technologies and the new distributed Perforce server features can be used to gain high availability and quick recoverability in different disaster scenarios. Handling large game content files and the dependencies between the game code and content assets will also be covered.

Steamlining your puppet development workflowTomas Doran

The document discusses ways to streamline a Puppet development workflow including using revision control, running Puppet in noop or automatic mode, moving changes slowly through testing and using branches, reporting on changes, and implementing testing strategies like unit testing with rspec-puppet and integration testing with serverspec. It also recommends tools like Foreman, Norman, Puppetfile, and Jenkins to improve testing and deployment.

ITP Spacebrew Workshop - Spring 2014Brett Renfer

Pushing a camel through the eye of a needleSensePost

RabbitMQ with python and ruby RuPy 2009Paolo Negri

Nodeconf npm 2011Florent Jaby ヅ

OpenNebulaConf 2014 - Lightning talk: OpenNebula Puppet Module - Norman Messt...OpenNebula Project

OpenNebula Conf 2014 | Lightning talk: OpenNebula Puppet Module - Norman Mess...NETWAYS

Spacebrew @ SFPCBrett Renfer

Automation with ansibleKhizer Naeem

Alvaro Videla, Building a Distributed Data Ingestion System with RabbitMQTanya Denisyuk

Encrypt your volumes with barbican open stack 2018Duncan Wannamaker

Построение распределенной системы сбора данных с помощью RabbitMQ, Alvaro Vid...Ontico

Firebird on LinuxMind The Firebird

IPv6 for Pentesterscamsec

IPv6 for PentestersNotSoSecure Global Services

Who Broke My CryptoJohn Varghese

Scaling Servers and Storage for Film Assets Perforce

Cooking a rabbit pieTomas Doran

Bh fed-03-kaminskyDan Kaminsky

Designing a Highly Available Environment Using Methods of Modern IT Infrastru...Perforce

Steamlining your puppet development workflowTomas Doran

More from PTIHPA (15)

Github:fi PresentationPTIHPA

Git is a distributed version control system that allows for flexible and secure collaboration. GitHub is a web-based hosting service for Git projects that makes collaboration easier through social networking and other project management features. The document provides instructions on installing Git and creating a GitHub account to get started using these tools for source code management and sharing projects.

2010 05 hands_onPTIHPA

The document provides instructions for using Putty for port forwarding to connect to a virtual machine running Windows XP and run VampirServer and VampirTrace. The steps include starting the virtual machine, starting PuTTY twice for connections, using one PuTTY window to set up port forwarding from the local machine to VampirServer running on the virtual machine, and using the second PuTTY window to connect through the port forwarding to run commands on the virtual machine. Once connected, running VampirServer and opening traces in the Vampir GUI is demonstrated.

Trace VisualizationPTIHPA

Vampir and VampirServer provide interactive visualization and analysis of trace data from parallel applications to help identify performance bottlenecks, with intuitive displays for browsing timeline, communication, and computation data across different levels of a parallel application from global to process to function. The toolset includes the VampirTrace instrumentation library and supports large trace files and high parallelism through a client-server architecture.

2010 02 instrumentation_and_runtime_measurementPTIHPA

VampirTrace provides instrumentation and run-time measurement capabilities. It allows for automatic, manual, and binary instrumentation. Run-time measurement includes collecting trace data behind the scenes and post-processing. Users have options to configure various settings like environment variables, hardware performance counters, memory allocation counters, filtering, and grouping. FAQ and troubleshooting information is also available.

2010 03 papi_indianaPTIHPA

The document discusses PAPI (Performance API), a tool for collecting hardware performance counter data from processors. PAPI provides a consistent interface for accessing performance counters across platforms and defines platform-neutral events. It supports many modern processors and operating systems and comes with utilities for collecting, analyzing, and visualizing performance data. An example shows how reordering loops in a matrix multiplication algorithm can improve data cache and TLB behavior.

Overview: Event Based Program AnalysisPTIHPA

Switc HpaPTIHPA

Digital data is increasingly being created and stored digitally rather than on physical tapes or drives. In the past, data was stored on tapes and transported between locations, resulting in data loss. Now, data is written directly to centralized storage systems and made accessible to researchers over networks from any location. Modern examples provided are seismic data from transportable sensors uploaded via cellular networks and medical images captured digitally and stored on a data capacitor system with backups. This infrastructure allows researchers to access and analyze large datasets from multiple locations without physical data transfer or loss.

Statewide It Robert HenschelPTIHPA

This document summarizes Robert Henschel's presentation about optimizing parallel applications to accelerate scientific discoveries. It discusses Indiana University's high performance computing resources like BigRed and Quarry as well as the TeraGrid. It provides examples of how the High Performance Applications group helped researchers integrate HPC systems into an electron microscope workflow, migrate simulations of gas giant planets to the TeraGrid, and develop computational models to predict drug interactions. The presentation aims to make researchers aware of computing resources and how to get research done more efficiently.

3 Vampir Trace In DetailPTIHPA

This document summarizes different aspects of instrumentation and runtime measurement using VampirTrace. It discusses automatic, manual, and binary instrumentation. It describes runtime measurement behind the scenes and using the OTF trace format. It outlines various options, settings, and parameters that can be configured using environment variables to control aspects like hardware counters, memory tracing, I/O tracing, and filtering.

2 Vampir Trace VisualizationPTIHPA

Web-based frontend to VampirServer Vampir for Windows VampirServer for Windows based on VampirServer architecture parallel server/client model Windows port of server components Windows client using Windows GUI toolkit Advantages platform independence scalable architecture leverage Windows GUI Challenges porting server components Windows GUI integration Status alpha version available active development Vampir Assistant Web-based frontend to VampirServer Advantages platform independence no installation easy access Features basic timeline visualization load, analyze, visualize traces share results Status prototype basic functionality active development

1 Vampir OverviewPTIHPA

4 HPA Examples Of Vampir UsagePTIHPA

Vampir is used at IU for summarizing and analyzing the performance of parallel applications in different domains. Examples include summarizing the performance of an image analysis pipeline with 640 repeated executions, analyzing a runtime anomaly on an IBM cluster, tracing applications on Cell BE systems, identifying a memory bottleneck in a particle swarm optimizer, finding an I/O issue in an EMAN application, understanding two applications for a cluster competition, tracing Pthreads in a Swarp application, and analyzing an MD simulation code parallelized with serial, OpenMP, MPI and hybrid approaches. Vampir provided insights into optimizations and issues across these different use cases.

GeneIndex: an open source parallel program for enumerating and locating words...PTIHPA

This document describes a parallel algorithm called GeneIndex for efficiently locating words within genomic sequences. It presents both a serial and parallel implementation. The serial algorithm scans sequences to encode and sort all possible words. The parallel implementation divides the input data and computation across processes. It aims to balance the workload by partitioning words rather than sequences, since word frequencies can vary greatly.

Implementing 3D SPHARM Surfaces Registration on Cell B.E. ProcessorPTIHPA

This document describes implementing 3D SPHARM surface registration on a Cell processor. It discusses SPHARM expansion and registration, calculating rotation coefficients, root mean square distance, and implementations in Matlab and on the Cell processor. The Cell implementation uses loop fusion, lookup tables, and optimizations for the Cell architecture like vectorization and data alignment. Performance analysis shows a dramatic increase in speed on the Cell due to its architecture and algorithm optimizations. Care is needed for data placement and transfer due to limited local store.

Big Iron and Parallel Processing, USArray Data Processing WorkshopPTIHPA

This document summarizes a presentation about big data processing systems and parallel computing techniques. It introduces several large supercomputing systems available through the TeraGrid, including Kraken, Ranger, and Abe. It discusses concepts like parallelism, load balancing, Amdahl's law, and scaling. It provides examples of using OpenMP and MPI for parallel programming and describes exercises to practice these techniques.

Github:fi PresentationPTIHPA

2010 05 hands_onPTIHPA

Trace VisualizationPTIHPA

2010 02 instrumentation_and_runtime_measurementPTIHPA

2010 03 papi_indianaPTIHPA

Overview: Event Based Program AnalysisPTIHPA

Switc HpaPTIHPA

Statewide It Robert HenschelPTIHPA

3 Vampir Trace In DetailPTIHPA

2 Vampir Trace VisualizationPTIHPA

1 Vampir OverviewPTIHPA

4 HPA Examples Of Vampir UsagePTIHPA

GeneIndex: an open source parallel program for enumerating and locating words...PTIHPA

Implementing 3D SPHARM Surfaces Registration on Cell B.E. ProcessorPTIHPA

Big Iron and Parallel Processing, USArray Data Processing WorkshopPTIHPA

2010 vampir workshop_iu_configuration

1. How To Use The Vampir Toolchain At IU Robert Henschel [email protected] April 2010

2. Contents • Quarry • BigRed Robert Henschel

3. General • Trace creation and visualization does not have to happen on the same platform! – You can look at traces created on Quarry (or your own desktop) on BigRed Robert Henschel

4. Quarry – SoftEnv Keys • Vampir Version 5.2 +vampir-5.2-32 or +vampir-5.2-64 • VampirServer Version 2 +vampirserver-2-openmpi-64 • VampirTrace Version 5.8 @intel-11.1.059-64 +openmpi-1.4.1-intel-11.1.059-64 +vampirtrace-5.8.0-intel-openmpi-64 or @intel-11.1.059-32 +openmpi-1.4.1-intel-11.1.059-32 +vampirtrace-5.8.0-intel-openmpi-32 Robert Henschel

5. Quarry – Running Vampir • Vampir Version 5.2 – Interactive job in PBS – On interactive nodes b005-b008 Robert Henschel

6. Quarry – Running Vampir • Vampir Version 5.2 – Interactive job in PBS qsub -I -X soft add +vampir-5.2-64 vampir – On interactive nodes b005-b008 • You share the node with other people, this is especially a problem for opening large trace files! ssh -X b005 soft add +vampir-5.2-64 vampir Robert Henschel

7. Quarry – Running VampirServer • Vampir Server Version 2 – Server in PBS, viewer on interactive node – Server and viewer in interactive job in PBS – Server and viewer on interactive nodes b005-b008 Robert Henschel

8. Quarry – Running VampirServer • Vampir Server Version 2 – Server in PBS, viewer on interactive node – Make sure your “.soft” file contains @intel-11.1.059-64 +openmpi-1.4.1-intel-11.1.059-64 +vampirserver-2-openmpi-64 – Logout/Login to active changes in “.soft” – PBS script #PBS -l nodes=2:ppn=8,walltime=5:00:00 #PBS -m ae #PBS -N VampirServer #PBS vampirserver --mode=mpi-mode --nproc=16 1>$HOME/vampir.out 2>&1 Start job qsub vampir.pbs – Wait for the job to start Robert Henschel

9. Quarry – Running VampirServer • Lookup host and port information » It may take a minute or two after the job started for this file to get updated!!! cat $HOME/vampir.out Running... Server listens on: b052:30000 • Viewer on interactive nodes b005-b008 ssh -X b005 soft add +openmpi-1.4.1-intel-11.1.059-64 soft add +vampirserver-2-openmpi-64 vampir //b052:30000/path/to/*.otf Robert Henschel

10. Quarry – Running VampirServer • Interactive job in PBS • Only one node qsub -I -X soft add +openmpi-1.4.1-intel-11.1.059-64 soft add +vampirserver-2-openmpi-64 vampirserver -m mpi-mode -n 8 1>output 2>&1 & cat output Running... Server listens on: b048:30000 vampir //b048:30000/path/to/*.otf Robert Henschel

11. Quarry – Running VampirServer On interactive nodes b005-b008 • Only one node • Runtime needs to be less than 24 hours • You share the node with other people ssh -X b005 soft add +openmpi-1.4.1-intel-11.1.059-64 soft add +vampirserver-2-openmpi-64 vampirserver -m mpi-mode -n 4 1>output 2>&1 & cat output Running... Server listens on: b005:30000 vampir //b005:30000/path/to/*.otf Robert Henschel

12. Quarry – Tracing with VampirTrace 5.8 • VampirTrace is configured to work with the Intel compilers version 11.1 and OpenMPI 1.4.1 • Include the following keys in your .soft file @intel-11.1.059-64 +openmpi-1.4.1-intel-11.1.059-64 +vampirtrace-5.8.0-intel-openmpi-64 • The VampirTrace compiler wrappers are now available • It is possible to use them also with GCC, if you need to do that, please send a mail to [email protected] • We can also provide a version for the PGI compilers. Robert Henschel

13. BigRed – SoftEnv Keys • Vampir Version 5.1 +vampir-5.1-64 • VampirServer Version 1.10 +vampirserver-1.10-64 • VampirTrace Version 5.6.3 +openmpi-1.3.1-ibm-64 +vampirtrace-5.6.3-ibm-64 or +openmpi-1.3.1-ibm-32 +vampirtrace-5.6.3-ibm-32 Robert Henschel

2010 vampir workshop_iu_configuration

Recommended

More Related Content

What's hot (20)

Similar to 2010 vampir workshop_iu_configuration (20)

More from PTIHPA (15)

2010 vampir workshop_iu_configuration