SlideShare a Scribd company logo

Remarks on Grids e-Science CyberInfrastructure and Peer-to-Peer ...

Download as PPT, PDF
••
V
Videoguy

NaradaBrokering is a messaging middleware that can reliably support grid messaging. It can virtualize inter-service communication, federate different grids, and scale to support large-scale applications like audio/video conferencing and collaborative applications. NaradaBrokering uses a broker-based architecture to allow scalable and reliable messaging across heterogeneous networks.

1 of 53
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
NaradaBrokering and GlobalMMCS 12 July 2004 Uniwaret Beihang University Peking Geoffrey Fox Community Grids Lab Indiana University [email_address]
NaradaBrokering can Support Grid Messaging reliably in spirit of WS-ReliableMessaging Virtualize inter-service communication Federate different Grids Together Scalable pervasive audio-video conferencing – “Video over IP” General collaborative Applications and Web services including e-learning, e-Sports and Internet multiplayer gaming Build next generation clients interacting with messages not method-based user interrupts Unify peer-to-peer networks and Grids Handle streams as in “media or sensor Grids” Handle events as in WS-Notification
NaradaBrokering Web Service B Stream Server-enhanced Messaging NB supports messages and streams NB role for Grid is Similar to MPI role for MPP Queues Minicomputer Firewall Computer Server PDA Modem Laptop computer Workstation Peers Peers Audio/Video Conferencing Client Audio/Video Conferencing Client NaradaBrokering Broker Network
“ GridMPI” v. NaradaBrokering In parallel computing , MPI and PVM provided “all the features one needed’ for inter-node messaging NB aims to play same role for the Grid but the requirements and constraints are very different NB is not MPI ported to a Grid/Globus environment Typically MPI aiming at microsecond latency but for Grid , time scales are different 100 millisecond quite normal network latency 30 millisecond typical packet time sensitivity (this is one audio or video frame) but even here can buffer 10-100 frames on client (conferencing to streaming) 1 millisecond is time for a Java server to “think” Jitter in latency (transit time through broker) due to routing, processing (in NB) or packet loss recovery is important property Grids need and can use software supported message functions and trade-offs between hardware and software routing different from parallel computing
NaradaBrokering Based on a network of cooperating broker nodes Cluster based architecture allows system to scale in size Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. Now has several core functions Reliable order-preserving “Optimized” Message transport (based on performance measurement) in heterogeneous multi-link fashion with TCP, UDP, SSL, HTTP, and will add GridFTP General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing Distributed XML event selection using XPATH metaphor QoS, Security profiles for sent and received messages Interface with reliable storage for persistent events
Laudable Features of NaradaBrokering Is open source http://www.naradabrokering.org Has client “plug-in” as well as standalone brokers Will have a discovery service to find nearest brokers Does tunnel through most firewalls without requiring ports to be opened Supports uniform time across a distributed network Supports JXTA, JMS (Java Message Service) and more powerful native mode Transit time < 1 millisecond per broker Will have setup and broker network administration module
NaradaBrokering Naturally Supports Filtering of events to support different client requirements (e.g,. PDA versus desktop, slow lines, different A/V codecs) Virtualization of addressing, routing, interfaces Federation and Mediation of multiple instances of Grid services as illustrated by Composition of Gridlets into full Grids (Gridlets are single computers in P2P case) JXTA with peer-group forming a Gridlet Monitoring of messages for Service management and general autonomic functions Fault tolerant data transport Virtual Private Grid with fine-grain Security model
Current NaradaBrokering Features Prototype implementation of WS-Notification Web Service Notifications Prototype implementation of WS-ReliableMessaging Web Service reliability NaradaBrokering enhanced Grid-FTP . Bridge to the Globus TK3 . Grid Application Support Java Message Service ( JMS ) 1.0.2b compliant Support for routing P2P JXTA interactions. Messaging Related Compliance Compression and Decompression of payloads Fragmentation a nd Coalescing of payloads Message Payload options Message-level WS-Security compatible security Security Recovery from failures and disconnects. Replay of events/messages at any time. Recovery and Replay Producer Order and Total Order over a message type Time Ordered delivery using Grid-wide NTP based absolute time Ordered delivery Robust and exactly-once delivery of messages in presence of failures Reliable delivery Subscription can be Strings, Integers, XPath queries, Regular Expressions , SQL and tag=value pairs. Subscription Formats Transport protocols supported include TCP,  Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS. Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing Multiple transport support In publish-subscribe Paradigm with different Protocols on each link
NaradaBrokering Service Integration Proxy Messaging Handler Messaging Notification Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's  gSOAP, Axis C++ S1 S2 P2 P1 S1 S2 S1 S2 S? Service P? Proxy NB Transport Standard SOAP Transport Any Transport
NB in the Transport Layer I Transport with optimizations for features such as performance, reliability, security using proxy or handler model Prototype of WS-RM (Reliable Messaging) using proxy GridFTP with NaradaBrokering transport (Parallel TCP plus WSRM) Messaging Substrate Consumer Service SOAP+HTTP Parallel TCP RTP …. Any Protocol satisfying QoS
NB in the Transport Layer II Some PDA’s have very asymmetric latency for Grid  PDA communication – we have designed a modified WSRM – WS-WRM wireless reliable messaging with different ack/nack choice Plan to support and federate WS-RM, WS-Reliability, WS-WRM NaradaBroker Filter 1 Filter 2 WS-RM WS-WRM WS-Reliability
PDA Latency Measurement These show high PDA latency
NB in the Transport Layer III We will add other higher performance protocols to NB transport options such as those based on UDP or modified TCP/IP We could support Virtual Private Network VPN to improve security (Virtual Private Grid) more choice on firewall/NAT tunneling Currently NB has a rich set of firewall penetration options but these are not yet fully packaged with correct strategy to use
Virtualizing Communication Communication specified in terms of user goal and Quality of Service – not in choice of port number and protocol Protocols have become overloaded e.g. MUST use UDP for A/V latency requirements but CAN’t use UDP as firewall will not support ……… A given communication can involve multiple transport protocols and multiple destinations – the latter possibly determined dynamically Dial-up Filter Satellite UDP Firewall HTTP A B 1 Hand-Held Protocol Fast Link Software Multicast B 2 B 3 NB Broker Client Filtering NB Brokers
Performance Monitoring Every broker incorporates a Monitoring service that monitors links originating from the node. Every link measures and exposes a set of metrics Average delays, jitters, loss rates, throughput. Individual links can disable measurements for individual or the entire set of metrics. Measurement intervals can also be varied Monitoring Service, returns measured metrics to Performance Aggregator .
Architecture of Message Layer Need to optimize not only routing of particular messages but classic publish/subscribe problem of integrating different requests with related topics (subscribe to sports/basketball/lakers and sports ) Related to Akamai, AOL … caching and Server optimization problem 1-> N Grid Clients Hypercube of NB Brokers (logical not physical) N≈100 for Distance Education Per edge Broker Scale with distributed Broker net?
NaradaBrokering Communication Applications interface to NaradaBrokering through UserChannels which NB constructs as a set of link s between NB Brokers acting as “ waystations” which may need to be dynamically instantiated UserChannels have publish/subscribe semantics with XML topics Link s implement a single conventional “data” protocol. Interface to add new transport protocols within the Framework Administrative channel negotiates the best available communication protocol for each link Different links can have different underlying transport implementations Implementations in the current release include support for TCP,UDP, Multicast, SSL, RTP and HTTP. GridFTP most interesting new protocol Supports communication through proxies and firewalls such as iPlanet , Netscape, Apache, Microsoft ISA and Checkpoint .
Pentium-3, 1GHz, 256 MB RAM 100 Mbps LAN JRE 1.3 Linux hop-3 0 1 2 3 4 5 6 7 8 9 100 1000 Transit Delay (Milliseconds) Message Payload Size (Bytes) Mean transit delay for message samples in NaradaBrokering: Different communication hops hop-2 hop-5 hop-7
 
 
 
NaradaBrokering and JMS (Java Message Service) (commercial JMS) Low Rate; Small Messages
NaradaBrokering and JXTA Federation Based on hybrid proxy that acts as both Rendezvous peer (JXTA routers) and NaradaBrokering end-point. No changes to JXTA core or constraints on interactions Change made to Rendezvous layer Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer. NB provides JXTA guaranteed long distance delivery NB federates multiple JXTA Peer Groups
End-point Services in Native NaradaBrokering Allows you to create Consumers (subscribers) of events (an event is a time stamped message where time stamp can be empty!) Allows you to create Producers of events (publishers) Allows you to discover brokers and initialize communications with the broker. Services available at the client side will perform Compression of payloads Computation of Message digests for Integrity Secure encryption of payload based on the specified keys Fragmentation of large payloads into smaller packets Redundancy service which maintains active (alternate) connections to multiple brokers.
Event Consumer Capabilities Allow you to subscribe to events that conform to a certain template. The specified subscription profile could topic-based strings, XPath queries, <tag=value> pairs or integer topics. Event Consumers can also create Consumer constraints to specify various properties regarding the delivery of events. Consumer constraints are different from subscriptions . Subscriptions (or Profiles ) are evaluated in a distributed fashion by the broker network, Consumer constraints are QoS related and are managed by the QoS services running on the end-point . Consumer constraints can specify Reliable Delivery of events Ordered (Publisher, causal and time ordered) delivery of events Exactly once delivery of events Delivery after un-compression of compressed payload Delivery after decrypting encrypted payload
Event Producer Capabilities Facilitate the generation of events in correct format (next slide) Facilitate the publishing of events to brokers Allow the creation of Publisher constraints which facilitate specification of properties that need to be satisfied by published events Among the constraints that can be specified include Method of Securing message payloads Computing message digests Compressing message payloads Fragmenting large payloads
Native NaradaBrokering Event The event comprises of Event headers Content Synopsis (for selection as in JMS properties WITHOUT reading body) Content Payload Dissemination Traces (generated on the fly as event traverses broker network) This is different from structure of JMS or JXTA events This NBEvent structure supports the extra capabilities discussed earlier The event headers specify information regarding Security and Integrity of encapsulated payload Fragmentation of events Compression of payloads Correlation identifiers (to define ordering between different streams as is needed in some collaboration applications ) Priority Application Type Event Identifiers
Based on Message Level Security Messages organized into topics Each topic has a separate key; Topics can be organized into sessions
Yes No Yes end-to-end Security Yes Yes No Support for XPath queries/ subscriptions Yes No Yes Communication through proxies and firewalls Yes No No Support for Audio/Video Conferencing & raw RTP clients JXTA and later Gnutella Yes No Support for routing P2P Interactions Yes Yes Yes Guaranteed Messaging (Robust) No Yes Medium (MQ is based on the point-to-point model. There is a limit on the effectiveness of this mode in large configurations). WebSphere MQ (formerly MQSeries) Yes No Network Performance Monitoring Yes No JMS Compliant Very large Very large Maximum number of nodes hosting the messaging infrastructure NaradaBrokering Pastry Functionality I
Yes No Yes Multiple transport protocols over multiple hops. TCP (Blocking and non-blocking), UDP, Multicast, HTTP, SSL, RTP, (GridFTP) TCP, UDP TCP, HTTP, Multicast, SSL, SNA etc. Transport Protocols Supported Fair with some “production” testing Fair Extremely mature , with very robust diagnostic information Maturity of Software Platforms supporting Java 1.4 (tunneling C++) Supported on platforms which support C# (Microsoft) or Java (Rice). 35 different OS/ platforms supported. Also supports the Java Platform. Platforms or Hosting Environments No Yes (Squirrel) No Support for P2P distributed caching No Yes WebSphere MQ (formerly MQSeries) In Progress No Broker Network Design Interface No No Workflow Support NaradaBrokering Pastry Functionality II
WS-Reliability & WS-RM There are two rival reliable messaging specifications for Web Services that provide reliable delivery between two endpoints. Both the specifications use positive acknowledgements to ensure reliable delivery Both specifications include support for faults WS-Reliability is a SOAP based protocol WS-ReliableMessaging provides an XML schema for reliable messaging. Includes a SOAP binding.
NaradaBrokering & Reliable Delivery specifications We can provide support for both these specifications In NaradaBrokering we provide reliable delivery from multiple points to multiple points We have identified issues that will allow federation between these specifications Sequence numbering, fault mappings, numbering rollovers, quality of service guarantees Federation would allow WSRM sender & WS-Reliability receiver WS-Reliability sender & WSRM receiver
NaradaBrokering, WS-Notification & JMS NaradaBrokering is JMS compliant Topics in NaradaBrokering could be based on XML, String(as in JMS), Plain text, Integers, and (tag=value) tuples. Subscriptions could be XPath queries, SQL queries, Regular expressions, Strings and integers Almost all the primitives needed in WS-Notification are available in NaradaBrokering Exception : Entities never communicate directly with each other, as proposed in WS-Notification. We are either allow such direct communication or mimic in NB – no performance overhead! We are currently building a prototype implementation of WS-Notification Need to relate WS-Notification with WS-Eventing and WS-Events
NaradaBrokering and NTP NaradaBrokering includes an implementation of the Network Time Protocol (NTP) All entities within the system use NTP to communicate with atomic time servers maintained by organizations like NIST and USNO to compute offsets Offset is the computed difference between global time and the local time. The offset is computed based on the time returned from multiple atomic time servers. The NTP algorithms weighs results from individual time clocks based on the distance of the atomic server from the entity. This ensures that all entities are within 1 millisecond of each other. The timestamps account for clock drifts that take place on machines Time returned is based on software clocks which can slow down with increased computing load on the machine.
Higher Level NB Capabilities Could presumably have a Perl Interface for WSRF:Lite Could federate between WS-Notification , JMS (which it already supports) and WS-Eventing (Microsoft) Using filters, it can be used to Reduce image size for PDA Convert Access Grid A/V to a form suitable for some PDA’s (RealNetworks, Windows Media) Move XML between SOAP header and body to federate between WS-RF and WS-I Could supply WS-Security if used in handler mode Could support replicated Subscriber (Fault-tolerance/ Performance) Services NaradaBrokering will choose between several subscribing replicated services
P2P and NaradaBrokering I Server/Broker-free version to support “immediate deployment” of NB-based Community Grids using P2P versions of Grid applications Initially: Use a broker free version of NB and file-based Web services If successful, add brokers in the Grid sky to achieve better performance (if broker has better network link than clients) Service providers and supercomputer/national grid centers could sell such Grid Farm services P2P Grid Farm in the Sky (clouds) NaradaBrokers
P2P and NaradaBrokering II Add DHT (Distributed Hash Table) approach (used in latest JXTA) to matching topics and subscriptions in NB Provides scalable matching of events between publishers and subscribers 3 models of Information Systems DHT for stable large volume distributed information Fault Tolerant Metadata Catalog – subscribe multiple instances of metadata service to the MetadataCatalog topic – publish queries to this replicated subscriber topic Flooding if all else has failed GridTorrent: Merge NB-enhanced GridFTP and P2P BitTorrent http://bitconjurer.org/BitTorrent/ to provide WSRM fault tolerant Parallel TCP P2P or Grid file transfer BitTorrent supports fragmented distributed files which are natural WSRM and NB architecture Don’t really want GridFTP server ; prefer to use fault tolerant GridTorrent metadata service (as above)
Fault Tolerant P2P e-Science Grid NaradaBrokering could provide several features of value to say particle physics Grid Supports Web Service standards: WS-RM, WS-Security, WS-Notification etc. Provide fault tolerant NaradaBroker network DHT provides worldwide scalable base information system Replicate all important services: RGMA, SRB, GRAM .. Associate each service with a topic Replicated services subscribe to topic Network and load QoS based choice of service GridTorrent file transfer automatically provides distributed fault tolerant caching – a better RLS
Collaboration and Web Services Collaboration has Mechanism to set up members (people, devices) of a “collaborative sessions” Shared generic tools such as text chat, white boards, audio-video conferencing Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments …. b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment We can port objects to Web Services and build a general approach for making Web services collaborative a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS
Shared Event Collaboration All collaboration is about sharing events defining state changes Audio/Video conferencing shares events specifying in compressed form audio or video Shared display shares events corresponding to change in pixels of a frame buffer Instant Messengers share updates to text message streams Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose update events of all kinds as message streams Need publish/subscrib e approach to share messages (NB) plus System to control “ session ” – who is collaborating and rules XGSP is XML protocol for controlling collaboration building on H323 and SIP
Web Services and M-MVC Web Services are naturally M-MVC – Message based Model View Controller with Model is Web Service Controller is Portal and Messages (NaradaBrokering) View is rendering As Controller
Desktop and Web Services with MMVC Most desktop applications are in fact roughly MVC with controller formed by “system interrupts” with View and Model communicating by “post an event” and define a “listener” programming mode We propose to integrate desktop and Web Service approach by systematic use of MMVC and NaradaBrokering Allows easier porting to diverse clients and automatic collaboration Attractive for next generation of Linux desktop clients We have demonstrated for SVG Browser (Scalable Vector Graphics), OpenOffice and PowerPoint “ Glob ” programming style makes hard
SM-MV Collaboration Shared Output port Single Model, Multiple View SM-MV Collaborative Web Service XGSP Session Control
MM-MV Collaboration Shared Input port Multiple Model, Multiple View MM-MV Collaborative Web Service
Global-MMCS Community Grid We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services. The function of A/V media server is distributed using NaradaBrokering architecture. Media Servers mix and convert A/V streams Open XGSP MCU based on the following open source projects openh323 is basis of H323 Gateway NIST SIP stack is basis of SIP Gateway NaradaBrokering is open source messaging Java Media Framework basis of Media Servers Helix Community http:// www.helixcommunity.org for Real Media http://www.globalmmcs.org open source “non advertised” release
XGSP Web Service MCU Architecture Gateways convert to uniform XGSP Messaging High Performance (RTP) and XML/SOAP and .. Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing NB Scales as distributed Web Services NaradaBrokering SIP H323 Access Grid Native XGSP Admire Media Servers Filters Session Server XGSP-based Control NaradaBrokering All Messaging
Break up into Web Services Monolithic MCU becomes many different “Simple Services” Session Control Thumbnail “image” grabber Audio Mixer Video Mixer Codec Conversion Helix Real Streaming PDA Conversion H323/SIP Gateways As independent can replicate particular services as needed Codec conversion might require 20 services for 20 streams spread over 5 machines 1000 simultaneous users could require: 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers Support with a stream optimized Grid Farm in the sky Future billion way “Video over IP” serving 3G Phones and home media centers/ TV ’s could require a lot of computing
GlobalMMCS and NaradaBrokering All communication – both control and “binary” codecs are handled by NaradaBrokering Control uses SOAP and codecs use RTP transport Each stream is regarded as a “topic” for NB Each RTP packet from this stream is regarded as an “event” for this topic Can use replay and persistency support in NB to support archiving and late clients Can build customized stream management to administer replay, and who gets what stream in what codec NaradaBrokering supports unicast and multicast Use firewall penetration and network monitoring services in NB to improve Q0S
0 10 20 30 40 50 60 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Delay (Milliseconds) Packet Number Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms NaradaBrokering-RTP JMF-RTP
0 1 2 3 4 5 6 7 8 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Jitter (Milliseconds) Packet Number Average jitter (std. dev) for 50 video clients. NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms NaradaBrokering-RTP JMF-RTP
Polycom, Access Grid and RealVideo views of video-mixed streams using GlobalMMCS
Integration of PDA, Cell phone and Desktop Grid Access NB Support for optimized PDA Communication
GlobalMMCS Futures Current “release” has very rudimentary session management Should support adding members and applications to a collaborative session Administration and master/non-master roles Collaborative PowerPoint , OpenOffice , SVG release waiting XGSP session manager Most interesting clients are Java applets supporting portlet model for modern portals Linux and Macintosh clients require higher performance JMF – Java Media Framework Need to support use of NB QoS features Add additional codecs like MPEG2 and MPEG4 Improve video codec-based shared display Need scheduler of dynamic services sensitive to streaming bandwidth requirement as well as CPU use of codec conversion

More Related Content

What's hot (18)

PDF
Week11 qo s-2017(1)
trayyoo
 
PPTX
Admission Control Mechanism For Mpls Ds Te
Optical solution, Network Adminstration, Software Developer
 
PPTX
Quality of service
Cesar Aguirre
 
PPTX
Protocols for internet of things
Charles Gibbons
 
PPTX
IAB-5039 : MQTT: A Protocol for the Internet of Things (InterConnect 2015)
PeterNiblett
 
DOC
vijaykumar CV (1)
vijaykumar veerapalli
 
PDF
An Introduction to the Message Queuning Technology
Harinath Krishnamoorthy
 
PDF
MSMQ - Microsoft Message Queueing
Peter R. Egli
 
PPT
Real Time Java DDS
kerush
 
PPTX
AndroidThing (Internet of things)
Mayur Solanki
 
PDF
Communication middleware
Peter R. Egli
 
PDF
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
David Ware
 
PDF
1463401 rc214-mqtt-update
Eugenio Lysei
 
PPT
Cliser
manjulrathi
 
PPTX
Message queuing telemetry transport (mqtt) message format
Hamdamboy (함담보이)
 
PPTX
Todays Mq Infrastructure And Tomorrows
gaborvodics
 
PDF
Networks
Alvaro Gomes
 
PDF
Towards an Open Data Center with an Interoperable Network (ODIN) : Volume 2: ...
IBM India Smarter Computing
 
Week11 qo s-2017(1)
trayyoo
 
Admission Control Mechanism For Mpls Ds Te
Optical solution, Network Adminstration, Software Developer
 
Quality of service
Cesar Aguirre
 
Protocols for internet of things
Charles Gibbons
 
IAB-5039 : MQTT: A Protocol for the Internet of Things (InterConnect 2015)
PeterNiblett
 
vijaykumar CV (1)
vijaykumar veerapalli
 
An Introduction to the Message Queuning Technology
Harinath Krishnamoorthy
 
MSMQ - Microsoft Message Queueing
Peter R. Egli
 
Real Time Java DDS
kerush
 
AndroidThing (Internet of things)
Mayur Solanki
 
Communication middleware
Peter R. Egli
 
IBM MQ: Managing Workloads, Scaling and Availability with MQ Clusters
David Ware
 
1463401 rc214-mqtt-update
Eugenio Lysei
 
Cliser
manjulrathi
 
Message queuing telemetry transport (mqtt) message format
Hamdamboy (함담보이)
 
Todays Mq Infrastructure And Tomorrows
gaborvodics
 
Networks
Alvaro Gomes
 
Towards an Open Data Center with an Interoperable Network (ODIN) : Volume 2: ...
IBM India Smarter Computing
 

Viewers also liked (8)

PPT
Frito-Lay Ideation Launchpad
Terry Crosby
 
PPT
Fritos Powerpoint
joscelynlasky
 
PDF
Frito Lay Recap PDF
tuellk12
 
PPT
Lays
Navpreet Ahuja
 
PPTX
Frito lay
Puneet Arora
 
PPTX
Frito lays critical decisions
Salma Malik
 
PPTX
Lays (Haram/Halal Campaign) 2009- ZK
Zareen Khan
 
PPTX
Из истории книги
Ольга Ивентьева
 
Frito-Lay Ideation Launchpad
Terry Crosby
 
Fritos Powerpoint
joscelynlasky
 
Frito Lay Recap PDF
tuellk12
 
Lays
Navpreet Ahuja
 
Frito lay
Puneet Arora
 
Frito lays critical decisions
Salma Malik
 
Lays (Haram/Halal Campaign) 2009- ZK
Zareen Khan
 
Из истории книги
Ольга Ивентьева
 
Ad

Similar to Remarks on Grids e-Science CyberInfrastructure and Peer-to-Peer ... (20)

PPT
ppt
Videoguy
 
PPT
NaradaBrokering Grid Messaging and Applications as Web Services
Videoguy
 
PPT
Collaboration and Grid Technologies
Videoguy
 
DOCX
Chapter 11 Selecting Technologies and Devices for Enterprise Netwo.docx
bartholomeocoombs
 
PPTX
Protocols for internet of things
Charles Gibbons
 
PPTX
Internet of Things: Protocols for M2M
Charles Gibbons
 
PPTX
Protocols for internet of things
Charles Gibbons
 
PPTX
Protocols for internet of things
Charles Gibbons
 
PPTX
Protocols for internet of things
Charles Gibbons
 
PPT
nv.ppt
tahaniali27
 
PPT
Implementation and Performance Analysis of a UDP Binding for SOAP
Dr. Fahad Aijaz
 
PPTX
Message Broker implementation in Kubernetes
Suman Chakraborty
 
PDF
Essay On Ethernet
Jennifer Wood
 
PDF
Messaging for IoT
Virtual JBoss User Group
 
PPTX
Design And Analysis Of MPLS based VPN
gandhimb
 
PPTX
Introduction to Kafka
Ducas Francis
 
PPTX
Unit 4.pptx
ssuser4b0b85
 
PPTX
Remote access service
Apoorw Pandey
 
PPTX
112321 112333 wirless application protocol
JAINIK PATEL
 
PPTX
Real Time Apps with SignalR Core
Sam Basu
 
ppt
Videoguy
 
NaradaBrokering Grid Messaging and Applications as Web Services
Videoguy
 
Collaboration and Grid Technologies
Videoguy
 
Chapter 11 Selecting Technologies and Devices for Enterprise Netwo.docx
bartholomeocoombs
 
Protocols for internet of things
Charles Gibbons
 
Internet of Things: Protocols for M2M
Charles Gibbons
 
Protocols for internet of things
Charles Gibbons
 
Protocols for internet of things
Charles Gibbons
 
Protocols for internet of things
Charles Gibbons
 
nv.ppt
tahaniali27
 
Implementation and Performance Analysis of a UDP Binding for SOAP
Dr. Fahad Aijaz
 
Message Broker implementation in Kubernetes
Suman Chakraborty
 
Essay On Ethernet
Jennifer Wood
 
Messaging for IoT
Virtual JBoss User Group
 
Design And Analysis Of MPLS based VPN
gandhimb
 
Introduction to Kafka
Ducas Francis
 
Unit 4.pptx
ssuser4b0b85
 
Remote access service
Apoorw Pandey
 
112321 112333 wirless application protocol
JAINIK PATEL
 
Real Time Apps with SignalR Core
Sam Basu
 
Ad

More from Videoguy (20)

PDF
Energy-Aware Wireless Video Streaming
Videoguy
 
PDF
Microsoft PowerPoint - WirelessCluster_Pres
Videoguy
 
PDF
Proxy Cache Management for Fine-Grained Scalable Video Streaming
Videoguy
 
PDF
Adobe
Videoguy
 
PDF
Free-riding Resilient Video Streaming in Peer-to-Peer Networks
Videoguy
 
PDF
Instant video streaming
Videoguy
 
PDF
Video Streaming over Bluetooth: A Survey
Videoguy
 
PDF
Video Streaming
Videoguy
 
PDF
Reaching a Broader Audience
Videoguy
 
PDF
Considerations for Creating Streamed Video Content over 3G ...
Videoguy
 
PDF
ADVANCES IN CHANNEL-ADAPTIVE VIDEO STREAMING
Videoguy
 
PDF
Impact of FEC Overhead on Scalable Video Streaming
Videoguy
 
PDF
Application Brief
Videoguy
 
PDF
Video Streaming Services – Stage 1
Videoguy
 
PDF
Streaming Video into Second Life
Videoguy
 
PDF
Flash Live Video Streaming Software
Videoguy
 
PDF
Videoconference Streaming Solutions Cookbook
Videoguy
 
PDF
Streaming Video Formaten
Videoguy
 
PDF
iPhone Live Video Streaming Software
Videoguy
 
PDF
Glow: Video streaming training guide - Firefox
Videoguy
 
Energy-Aware Wireless Video Streaming
Videoguy
 
Microsoft PowerPoint - WirelessCluster_Pres
Videoguy
 
Proxy Cache Management for Fine-Grained Scalable Video Streaming
Videoguy
 
Adobe
Videoguy
 
Free-riding Resilient Video Streaming in Peer-to-Peer Networks
Videoguy
 
Instant video streaming
Videoguy
 
Video Streaming over Bluetooth: A Survey
Videoguy
 
Video Streaming
Videoguy
 
Reaching a Broader Audience
Videoguy
 
Considerations for Creating Streamed Video Content over 3G ...
Videoguy
 
ADVANCES IN CHANNEL-ADAPTIVE VIDEO STREAMING
Videoguy
 
Impact of FEC Overhead on Scalable Video Streaming
Videoguy
 
Application Brief
Videoguy
 
Video Streaming Services – Stage 1
Videoguy
 
Streaming Video into Second Life
Videoguy
 
Flash Live Video Streaming Software
Videoguy
 
Videoconference Streaming Solutions Cookbook
Videoguy
 
Streaming Video Formaten
Videoguy
 
iPhone Live Video Streaming Software
Videoguy
 
Glow: Video streaming training guide - Firefox
Videoguy
 

Remarks on Grids e-Science CyberInfrastructure and Peer-to-Peer ...

  • 1. NaradaBrokering and GlobalMMCS 12 July 2004 Uniwaret Beihang University Peking Geoffrey Fox Community Grids Lab Indiana University [email_address]
  • 2. NaradaBrokering can Support Grid Messaging reliably in spirit of WS-ReliableMessaging Virtualize inter-service communication Federate different Grids Together Scalable pervasive audio-video conferencing – “Video over IP” General collaborative Applications and Web services including e-learning, e-Sports and Internet multiplayer gaming Build next generation clients interacting with messages not method-based user interrupts Unify peer-to-peer networks and Grids Handle streams as in “media or sensor Grids” Handle events as in WS-Notification
  • 3. NaradaBrokering Web Service B Stream Server-enhanced Messaging NB supports messages and streams NB role for Grid is Similar to MPI role for MPP Queues Minicomputer Firewall Computer Server PDA Modem Laptop computer Workstation Peers Peers Audio/Video Conferencing Client Audio/Video Conferencing Client NaradaBrokering Broker Network
  • 4. “ GridMPI” v. NaradaBrokering In parallel computing , MPI and PVM provided “all the features one needed’ for inter-node messaging NB aims to play same role for the Grid but the requirements and constraints are very different NB is not MPI ported to a Grid/Globus environment Typically MPI aiming at microsecond latency but for Grid , time scales are different 100 millisecond quite normal network latency 30 millisecond typical packet time sensitivity (this is one audio or video frame) but even here can buffer 10-100 frames on client (conferencing to streaming) 1 millisecond is time for a Java server to “think” Jitter in latency (transit time through broker) due to routing, processing (in NB) or packet loss recovery is important property Grids need and can use software supported message functions and trade-offs between hardware and software routing different from parallel computing
  • 5. NaradaBrokering Based on a network of cooperating broker nodes Cluster based architecture allows system to scale in size Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. Now has several core functions Reliable order-preserving “Optimized” Message transport (based on performance measurement) in heterogeneous multi-link fashion with TCP, UDP, SSL, HTTP, and will add GridFTP General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing Distributed XML event selection using XPATH metaphor QoS, Security profiles for sent and received messages Interface with reliable storage for persistent events
  • 6. Laudable Features of NaradaBrokering Is open source http://www.naradabrokering.org Has client “plug-in” as well as standalone brokers Will have a discovery service to find nearest brokers Does tunnel through most firewalls without requiring ports to be opened Supports uniform time across a distributed network Supports JXTA, JMS (Java Message Service) and more powerful native mode Transit time < 1 millisecond per broker Will have setup and broker network administration module
  • 7. NaradaBrokering Naturally Supports Filtering of events to support different client requirements (e.g,. PDA versus desktop, slow lines, different A/V codecs) Virtualization of addressing, routing, interfaces Federation and Mediation of multiple instances of Grid services as illustrated by Composition of Gridlets into full Grids (Gridlets are single computers in P2P case) JXTA with peer-group forming a Gridlet Monitoring of messages for Service management and general autonomic functions Fault tolerant data transport Virtual Private Grid with fine-grain Security model
  • 8. Current NaradaBrokering Features Prototype implementation of WS-Notification Web Service Notifications Prototype implementation of WS-ReliableMessaging Web Service reliability NaradaBrokering enhanced Grid-FTP . Bridge to the Globus TK3 . Grid Application Support Java Message Service ( JMS ) 1.0.2b compliant Support for routing P2P JXTA interactions. Messaging Related Compliance Compression and Decompression of payloads Fragmentation a nd Coalescing of payloads Message Payload options Message-level WS-Security compatible security Security Recovery from failures and disconnects. Replay of events/messages at any time. Recovery and Replay Producer Order and Total Order over a message type Time Ordered delivery using Grid-wide NTP based absolute time Ordered delivery Robust and exactly-once delivery of messages in presence of failures Reliable delivery Subscription can be Strings, Integers, XPath queries, Regular Expressions , SQL and tag=value pairs. Subscription Formats Transport protocols supported include TCP,  Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS. Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing Multiple transport support In publish-subscribe Paradigm with different Protocols on each link
  • 9. NaradaBrokering Service Integration Proxy Messaging Handler Messaging Notification Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's  gSOAP, Axis C++ S1 S2 P2 P1 S1 S2 S1 S2 S? Service P? Proxy NB Transport Standard SOAP Transport Any Transport
  • 10. NB in the Transport Layer I Transport with optimizations for features such as performance, reliability, security using proxy or handler model Prototype of WS-RM (Reliable Messaging) using proxy GridFTP with NaradaBrokering transport (Parallel TCP plus WSRM) Messaging Substrate Consumer Service SOAP+HTTP Parallel TCP RTP …. Any Protocol satisfying QoS
  • 11. NB in the Transport Layer II Some PDA’s have very asymmetric latency for Grid  PDA communication – we have designed a modified WSRM – WS-WRM wireless reliable messaging with different ack/nack choice Plan to support and federate WS-RM, WS-Reliability, WS-WRM NaradaBroker Filter 1 Filter 2 WS-RM WS-WRM WS-Reliability
  • 12. PDA Latency Measurement These show high PDA latency
  • 13. NB in the Transport Layer III We will add other higher performance protocols to NB transport options such as those based on UDP or modified TCP/IP We could support Virtual Private Network VPN to improve security (Virtual Private Grid) more choice on firewall/NAT tunneling Currently NB has a rich set of firewall penetration options but these are not yet fully packaged with correct strategy to use
  • 14. Virtualizing Communication Communication specified in terms of user goal and Quality of Service – not in choice of port number and protocol Protocols have become overloaded e.g. MUST use UDP for A/V latency requirements but CAN’t use UDP as firewall will not support ……… A given communication can involve multiple transport protocols and multiple destinations – the latter possibly determined dynamically Dial-up Filter Satellite UDP Firewall HTTP A B 1 Hand-Held Protocol Fast Link Software Multicast B 2 B 3 NB Broker Client Filtering NB Brokers
  • 15. Performance Monitoring Every broker incorporates a Monitoring service that monitors links originating from the node. Every link measures and exposes a set of metrics Average delays, jitters, loss rates, throughput. Individual links can disable measurements for individual or the entire set of metrics. Measurement intervals can also be varied Monitoring Service, returns measured metrics to Performance Aggregator .
  • 16. Architecture of Message Layer Need to optimize not only routing of particular messages but classic publish/subscribe problem of integrating different requests with related topics (subscribe to sports/basketball/lakers and sports ) Related to Akamai, AOL … caching and Server optimization problem 1-> N Grid Clients Hypercube of NB Brokers (logical not physical) N≈100 for Distance Education Per edge Broker Scale with distributed Broker net?
  • 17. NaradaBrokering Communication Applications interface to NaradaBrokering through UserChannels which NB constructs as a set of link s between NB Brokers acting as “ waystations” which may need to be dynamically instantiated UserChannels have publish/subscribe semantics with XML topics Link s implement a single conventional “data” protocol. Interface to add new transport protocols within the Framework Administrative channel negotiates the best available communication protocol for each link Different links can have different underlying transport implementations Implementations in the current release include support for TCP,UDP, Multicast, SSL, RTP and HTTP. GridFTP most interesting new protocol Supports communication through proxies and firewalls such as iPlanet , Netscape, Apache, Microsoft ISA and Checkpoint .
  • 18. Pentium-3, 1GHz, 256 MB RAM 100 Mbps LAN JRE 1.3 Linux hop-3 0 1 2 3 4 5 6 7 8 9 100 1000 Transit Delay (Milliseconds) Message Payload Size (Bytes) Mean transit delay for message samples in NaradaBrokering: Different communication hops hop-2 hop-5 hop-7
  • 19.  
  • 20.  
  • 21.  
  • 22. NaradaBrokering and JMS (Java Message Service) (commercial JMS) Low Rate; Small Messages
  • 23. NaradaBrokering and JXTA Federation Based on hybrid proxy that acts as both Rendezvous peer (JXTA routers) and NaradaBrokering end-point. No changes to JXTA core or constraints on interactions Change made to Rendezvous layer Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer. NB provides JXTA guaranteed long distance delivery NB federates multiple JXTA Peer Groups
  • 24. End-point Services in Native NaradaBrokering Allows you to create Consumers (subscribers) of events (an event is a time stamped message where time stamp can be empty!) Allows you to create Producers of events (publishers) Allows you to discover brokers and initialize communications with the broker. Services available at the client side will perform Compression of payloads Computation of Message digests for Integrity Secure encryption of payload based on the specified keys Fragmentation of large payloads into smaller packets Redundancy service which maintains active (alternate) connections to multiple brokers.
  • 25. Event Consumer Capabilities Allow you to subscribe to events that conform to a certain template. The specified subscription profile could topic-based strings, XPath queries, <tag=value> pairs or integer topics. Event Consumers can also create Consumer constraints to specify various properties regarding the delivery of events. Consumer constraints are different from subscriptions . Subscriptions (or Profiles ) are evaluated in a distributed fashion by the broker network, Consumer constraints are QoS related and are managed by the QoS services running on the end-point . Consumer constraints can specify Reliable Delivery of events Ordered (Publisher, causal and time ordered) delivery of events Exactly once delivery of events Delivery after un-compression of compressed payload Delivery after decrypting encrypted payload
  • 26. Event Producer Capabilities Facilitate the generation of events in correct format (next slide) Facilitate the publishing of events to brokers Allow the creation of Publisher constraints which facilitate specification of properties that need to be satisfied by published events Among the constraints that can be specified include Method of Securing message payloads Computing message digests Compressing message payloads Fragmenting large payloads
  • 27. Native NaradaBrokering Event The event comprises of Event headers Content Synopsis (for selection as in JMS properties WITHOUT reading body) Content Payload Dissemination Traces (generated on the fly as event traverses broker network) This is different from structure of JMS or JXTA events This NBEvent structure supports the extra capabilities discussed earlier The event headers specify information regarding Security and Integrity of encapsulated payload Fragmentation of events Compression of payloads Correlation identifiers (to define ordering between different streams as is needed in some collaboration applications ) Priority Application Type Event Identifiers
  • 28. Based on Message Level Security Messages organized into topics Each topic has a separate key; Topics can be organized into sessions
  • 29. Yes No Yes end-to-end Security Yes Yes No Support for XPath queries/ subscriptions Yes No Yes Communication through proxies and firewalls Yes No No Support for Audio/Video Conferencing & raw RTP clients JXTA and later Gnutella Yes No Support for routing P2P Interactions Yes Yes Yes Guaranteed Messaging (Robust) No Yes Medium (MQ is based on the point-to-point model. There is a limit on the effectiveness of this mode in large configurations). WebSphere MQ (formerly MQSeries) Yes No Network Performance Monitoring Yes No JMS Compliant Very large Very large Maximum number of nodes hosting the messaging infrastructure NaradaBrokering Pastry Functionality I
  • 30. Yes No Yes Multiple transport protocols over multiple hops. TCP (Blocking and non-blocking), UDP, Multicast, HTTP, SSL, RTP, (GridFTP) TCP, UDP TCP, HTTP, Multicast, SSL, SNA etc. Transport Protocols Supported Fair with some “production” testing Fair Extremely mature , with very robust diagnostic information Maturity of Software Platforms supporting Java 1.4 (tunneling C++) Supported on platforms which support C# (Microsoft) or Java (Rice). 35 different OS/ platforms supported. Also supports the Java Platform. Platforms or Hosting Environments No Yes (Squirrel) No Support for P2P distributed caching No Yes WebSphere MQ (formerly MQSeries) In Progress No Broker Network Design Interface No No Workflow Support NaradaBrokering Pastry Functionality II
  • 31. WS-Reliability & WS-RM There are two rival reliable messaging specifications for Web Services that provide reliable delivery between two endpoints. Both the specifications use positive acknowledgements to ensure reliable delivery Both specifications include support for faults WS-Reliability is a SOAP based protocol WS-ReliableMessaging provides an XML schema for reliable messaging. Includes a SOAP binding.
  • 32. NaradaBrokering & Reliable Delivery specifications We can provide support for both these specifications In NaradaBrokering we provide reliable delivery from multiple points to multiple points We have identified issues that will allow federation between these specifications Sequence numbering, fault mappings, numbering rollovers, quality of service guarantees Federation would allow WSRM sender & WS-Reliability receiver WS-Reliability sender & WSRM receiver
  • 33. NaradaBrokering, WS-Notification & JMS NaradaBrokering is JMS compliant Topics in NaradaBrokering could be based on XML, String(as in JMS), Plain text, Integers, and (tag=value) tuples. Subscriptions could be XPath queries, SQL queries, Regular expressions, Strings and integers Almost all the primitives needed in WS-Notification are available in NaradaBrokering Exception : Entities never communicate directly with each other, as proposed in WS-Notification. We are either allow such direct communication or mimic in NB – no performance overhead! We are currently building a prototype implementation of WS-Notification Need to relate WS-Notification with WS-Eventing and WS-Events
  • 34. NaradaBrokering and NTP NaradaBrokering includes an implementation of the Network Time Protocol (NTP) All entities within the system use NTP to communicate with atomic time servers maintained by organizations like NIST and USNO to compute offsets Offset is the computed difference between global time and the local time. The offset is computed based on the time returned from multiple atomic time servers. The NTP algorithms weighs results from individual time clocks based on the distance of the atomic server from the entity. This ensures that all entities are within 1 millisecond of each other. The timestamps account for clock drifts that take place on machines Time returned is based on software clocks which can slow down with increased computing load on the machine.
  • 35. Higher Level NB Capabilities Could presumably have a Perl Interface for WSRF:Lite Could federate between WS-Notification , JMS (which it already supports) and WS-Eventing (Microsoft) Using filters, it can be used to Reduce image size for PDA Convert Access Grid A/V to a form suitable for some PDA’s (RealNetworks, Windows Media) Move XML between SOAP header and body to federate between WS-RF and WS-I Could supply WS-Security if used in handler mode Could support replicated Subscriber (Fault-tolerance/ Performance) Services NaradaBrokering will choose between several subscribing replicated services
  • 36. P2P and NaradaBrokering I Server/Broker-free version to support “immediate deployment” of NB-based Community Grids using P2P versions of Grid applications Initially: Use a broker free version of NB and file-based Web services If successful, add brokers in the Grid sky to achieve better performance (if broker has better network link than clients) Service providers and supercomputer/national grid centers could sell such Grid Farm services P2P Grid Farm in the Sky (clouds) NaradaBrokers
  • 37. P2P and NaradaBrokering II Add DHT (Distributed Hash Table) approach (used in latest JXTA) to matching topics and subscriptions in NB Provides scalable matching of events between publishers and subscribers 3 models of Information Systems DHT for stable large volume distributed information Fault Tolerant Metadata Catalog – subscribe multiple instances of metadata service to the MetadataCatalog topic – publish queries to this replicated subscriber topic Flooding if all else has failed GridTorrent: Merge NB-enhanced GridFTP and P2P BitTorrent http://bitconjurer.org/BitTorrent/ to provide WSRM fault tolerant Parallel TCP P2P or Grid file transfer BitTorrent supports fragmented distributed files which are natural WSRM and NB architecture Don’t really want GridFTP server ; prefer to use fault tolerant GridTorrent metadata service (as above)
  • 38. Fault Tolerant P2P e-Science Grid NaradaBrokering could provide several features of value to say particle physics Grid Supports Web Service standards: WS-RM, WS-Security, WS-Notification etc. Provide fault tolerant NaradaBroker network DHT provides worldwide scalable base information system Replicate all important services: RGMA, SRB, GRAM .. Associate each service with a topic Replicated services subscribe to topic Network and load QoS based choice of service GridTorrent file transfer automatically provides distributed fault tolerant caching – a better RLS
  • 39. Collaboration and Web Services Collaboration has Mechanism to set up members (people, devices) of a “collaborative sessions” Shared generic tools such as text chat, white boards, audio-video conferencing Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments …. b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment We can port objects to Web Services and build a general approach for making Web services collaborative a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS
  • 40. Shared Event Collaboration All collaboration is about sharing events defining state changes Audio/Video conferencing shares events specifying in compressed form audio or video Shared display shares events corresponding to change in pixels of a frame buffer Instant Messengers share updates to text message streams Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose update events of all kinds as message streams Need publish/subscrib e approach to share messages (NB) plus System to control “ session ” – who is collaborating and rules XGSP is XML protocol for controlling collaboration building on H323 and SIP
  • 41. Web Services and M-MVC Web Services are naturally M-MVC – Message based Model View Controller with Model is Web Service Controller is Portal and Messages (NaradaBrokering) View is rendering As Controller
  • 42. Desktop and Web Services with MMVC Most desktop applications are in fact roughly MVC with controller formed by “system interrupts” with View and Model communicating by “post an event” and define a “listener” programming mode We propose to integrate desktop and Web Service approach by systematic use of MMVC and NaradaBrokering Allows easier porting to diverse clients and automatic collaboration Attractive for next generation of Linux desktop clients We have demonstrated for SVG Browser (Scalable Vector Graphics), OpenOffice and PowerPoint “ Glob ” programming style makes hard
  • 43. SM-MV Collaboration Shared Output port Single Model, Multiple View SM-MV Collaborative Web Service XGSP Session Control
  • 44. MM-MV Collaboration Shared Input port Multiple Model, Multiple View MM-MV Collaborative Web Service
  • 45. Global-MMCS Community Grid We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services. The function of A/V media server is distributed using NaradaBrokering architecture. Media Servers mix and convert A/V streams Open XGSP MCU based on the following open source projects openh323 is basis of H323 Gateway NIST SIP stack is basis of SIP Gateway NaradaBrokering is open source messaging Java Media Framework basis of Media Servers Helix Community http:// www.helixcommunity.org for Real Media http://www.globalmmcs.org open source “non advertised” release
  • 46. XGSP Web Service MCU Architecture Gateways convert to uniform XGSP Messaging High Performance (RTP) and XML/SOAP and .. Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing NB Scales as distributed Web Services NaradaBrokering SIP H323 Access Grid Native XGSP Admire Media Servers Filters Session Server XGSP-based Control NaradaBrokering All Messaging
  • 47. Break up into Web Services Monolithic MCU becomes many different “Simple Services” Session Control Thumbnail “image” grabber Audio Mixer Video Mixer Codec Conversion Helix Real Streaming PDA Conversion H323/SIP Gateways As independent can replicate particular services as needed Codec conversion might require 20 services for 20 streams spread over 5 machines 1000 simultaneous users could require: 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers Support with a stream optimized Grid Farm in the sky Future billion way “Video over IP” serving 3G Phones and home media centers/ TV ’s could require a lot of computing
  • 48. GlobalMMCS and NaradaBrokering All communication – both control and “binary” codecs are handled by NaradaBrokering Control uses SOAP and codecs use RTP transport Each stream is regarded as a “topic” for NB Each RTP packet from this stream is regarded as an “event” for this topic Can use replay and persistency support in NB to support archiving and late clients Can build customized stream management to administer replay, and who gets what stream in what codec NaradaBrokering supports unicast and multicast Use firewall penetration and network monitoring services in NB to improve Q0S
  • 49. 0 10 20 30 40 50 60 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Delay (Milliseconds) Packet Number Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms NaradaBrokering-RTP JMF-RTP
  • 50. 0 1 2 3 4 5 6 7 8 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Jitter (Milliseconds) Packet Number Average jitter (std. dev) for 50 video clients. NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms NaradaBrokering-RTP JMF-RTP
  • 51. Polycom, Access Grid and RealVideo views of video-mixed streams using GlobalMMCS
  • 52. Integration of PDA, Cell phone and Desktop Grid Access NB Support for optimized PDA Communication
  • 53. GlobalMMCS Futures Current “release” has very rudimentary session management Should support adding members and applications to a collaborative session Administration and master/non-master roles Collaborative PowerPoint , OpenOffice , SVG release waiting XGSP session manager Most interesting clients are Java applets supporting portlet model for modern portals Linux and Macintosh clients require higher performance JMF – Java Media Framework Need to support use of NB QoS features Add additional codecs like MPEG2 and MPEG4 Improve video codec-based shared display Need scheduler of dynamic services sensitive to streaming bandwidth requirement as well as CPU use of codec conversion

Editor's Notes

  • #16: Every broker in NaradaBrokering incorporates a monitoring service (as shown in Figure 12) that monitors the state of the links originating from the broker node. Metrics computed and reported over individual links, originating from a broker node, include bandwidth , jitter , transit delays , loss rates and system throughputs . Factors are measured in a non-intrusive way so as to ensure that the measurements do not further degrade the metrics being measured in the first place. Factors such as bandwidth measurements, which can pollute other metrics being measured, are measured at lesser frequencies. Furthermore, once a link is deemed to be at the extreme ends of the performance spectrum (either very good or very bad) the measurement of certain factors are turned off while others are measured at a far lower frequency . The monitoring service then reports this data to a performance aggregator node, which aggregates information from monitoring services running at other nodes.
  • #24: The integration is based on the proxy model, which essentially acts as the bridge between the NaradaBrokering system and JXTA. The Narada-JXTA proxy, operating inside the JXTA rendezvous layer, serves in a dual role as both a rendezvous peer and as a NaradaBrokering client providing a bridge between NaradaBrokering and JXTA. NaradaBrokering could be viewed as a service by JXTA. We make no changes to the JXTA core and the associated protocols. We make additions to the rendezvous layer for integration purposes. Second, this integration should entail neither any changes to the peers nor a restriction of the interactions that these peers could have had prior to the integration. Peers do not know that the broker network is routing some of their interactions. Furthermore, these Narada-JXTA proxies, since they are configured as clients within the NaradaBrokering system, inherit all the guarantees that are provided to NaradaBrokering clients.
  • #29: The figure above depicts the sequence of operations involved in securing messaging within NB. When an entity requests permission to publish, the KMC responds back with a topic key if the entity is authorized to publish to the topic. The entity then proceeds to encrypt the message with the topic key, compute a message digest and sign the message digest with its personal private-key. Individual Brokers upon receipt of the secure message can verify the entity signatures and permissions. If it is not authorized to publish or if the message integrity is compromised (revealed by the message digest) the message is discarded. When an entity requests permission to subscribe to a topic, the KMC returns it the topic keys if the entity is authorized to subscribe. Since the subscription needs to be propagated within the system, the entity propagates its subscription request by signing the request with its personal public-key. Upon receipt of a secure message, the subscriber verifies the signature and integrity of the message. It then proceeds to decrypt the secure message with the KMC supplied secret topic-key.
  • #50: Standarad deviation between inter packet arrival times