Ad
More Related Content
What's hot (20)
Similar to What's New Overview for IBM Streams V4.3 (20)
Ad
More from lisanl (20)
Ad
Recently uploaded (20)
What's New Overview for IBM Streams V4.3
- 1. © 2018 IBM Corporation1 Streams V4.3 What’s New Overview August 2018 Mike Spicer Distinguished Engineer, Lead Architect - IBM Streams
- 2. © 2018 IBM Corporation2 Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 7/30/20182
- 3. © 2018 IBM Corporation3 © 2018 IBM Corporation3 Serverless Workloads • Deploy applications without pre-assigning resources to a Streams Instance • Resources freed when applications are cancelled • Shared and exclusive assignment simplifies workload Isolation and fine grain resource allocation Dynamic User Defined Parallelism (UDP) • Dynamically change the width of a parallel region at runtime • Dynamically change the resources available to the parallel region • The Streams Scheduler may change the fusion of operators when the width is changed Roadmap • A first step on the path to being fully dynamic • Manual commands and “hooks” with a number of restrictions • We plan to increase function and remove restrictions over future releases • Goal is to have seamless and automatic capabilities with minimal constraints and impacts on applications • Other systems have constraints and implications • Stateless or Immutable state; • Lack of ordering or processing guarantees; • High overhead or latency impact; Dynamic and Elastic Scaling
- 4. © 2018 IBM Corporation4 © 2018 IBM Corporation4 SPL Optional Data Type • Allows an attribute to have no value, similar to SQL NULL • Improves interactions with external systems that support the concept of data fields which have no value • Changes to SPL and a number of toolkits SPL Event-Time Window Processing • Simplifies support for window operations based on the time of an event • Tuples may include an event time attribute that is different from the time it is received or processed • Simplifies support for out of order and late arriving data • New SPL window type and stream annotation Java and Python Topology API • A number of improvements developed in the open at http://ibmstreams.github.io/streamsx.topology/ Programming Model
- 5. © 2018 IBM Corporation5 © 2018 IBM Corporation5 Support for Beam-2.4 Java SDK • Run Beam applications on your local Streams instance • Use Beam's S3 IO connector to store results to an IBM Cloud Object Storage service Parallel Beam Pipelines and Transforms • Specify parallel widths for your application pipeline or specific Beam transforms at compile time Tuple Bundling • Specify bundle sizes or time limit at compile time Componentized Logging • Configure translation-time and runtime trace levels separately Submission-Time Parameters • Create submission-time parameters using Beam application options Streams Runner for Apache Beam New Features
- 6. © 2018 IBM Corporation6 © 2018 IBM Corporation6 Improved Scheduling • Make decisions based on additional data • Network Load Consideration in PE Placement • Allow users to control the relative importance of different data for their applications and environment • Multi Dimensional User Selectable Scheduling Scheme Improved Rolling Upgrades • Instances can be upgraded without stopping all applications and restarting the instance Operations & System Management 6
- 7. © 2018 IBM Corporation7 © 2018 IBM Corporation7 New toolkits: • Kafka, MessageHub, MQTT, RabbitMQ (primarily breaking out from streams.messaging toolkit) • Cloud Object Storage Toolkit extensions (new operators): • Geospatial • FlightPathEncounter • Timeseries • BATS forecasting, Holt Winters (new version) • Inet • InetSource, HTTPGetStream, HTTPPost, HTTPGetJSONContent, HTTPGetXMLContent, HTTPRequest Toolkits 7
- 8. © 2018 IBM Corporation8 Thank You