The Eclipse Transformer Project

Editor's Notes

• #2: (15 min before presentation follow presentation link) Hello! I am Thomas Bitonti. Together with BJ Hargrave, I’d like to welcome you to this presentation, which is all about the Eclipse Transformer.
• #3: Both BJ Hargrave and I have extensive experience as software developers at IBM, and are the co-authors of the Eclipse Transformer.
• #4: This presentation provides an overview of the Eclipse Transformer, telling what problem the transformer solves, and for whom. The presentation provides details and examples of the updates which are made by the transformer, and shows how those updates are configured. The presentation provides a brief history of the development of the transformer, and notes how the transformer may be extended for additional use. This presentation concludes with several useful links, including links to the transformer itself and including links to the most recent rules data which is used for JavaEE to Jakarta transformations.
• #5: The Eclipse Transformer is an Eclipse hosted open source project which provides a tool for performing class and JAR shading, and which, by performing package replacement, may be used to transform existing JavaEE classes and archives into JakartaEE compatible equivalents. Two comprehensive capabilities which are provided are the transformation of OSGi bundle archives and the transformation entire JavaEE applications.
• #6: (Pause)
• #7: (Pause)
• #8: A core problem facing developers who are working within the Jakarta space is to replace JavaEE package references with Jakarta package references. This core problem occurs with many types of JavaEE artifacts, starting with java classes, and quickly extending to other artifacts, such as manifests, deployment descriptors, and server configurations, and extending to standard JavaEE archives. The core problem is seen when the goal is to create new Jakarta artifacts starting with JavaEE artifacts. (next)
• #9: The need to create Jakarta artifacts is problem for a variety of developers – server developers, test developers, and application developers.
• #10: Initial use has been by the Open Liberty, WildFly, and tomitribe, whose development organizations have used the transformer to add Jakarta support to their existing JavaEE offerings.
• #11: Looking forward, the predominant usage is envisioned to be first, the creation of Jakarta versions of existing JavaEE applications, and second, the assembly of new Jakarta applications from existing JavaEE components with new Jakarta components. Already, as noted in other LiveStream presentation, Payara uses the transformer to dynamically transform artifacts during application deployment.
• #12: The value statement of the Eclipse Transformer is that it enables the rapid creation of Jakarta artifacts from existing JavaEE artifacts. When using the transformer, existing JavaEE artifacts need not be updated at the source level and rebuilt. Instead, new Jakarta artifacts in a single step from existing JavaEE artifacts. The Eclipse Transformer is a nut-and-bolts tool, designed to solve a practical but widespread problem. Our goal is that the transformer will assist developers to more seamlessly create new Jakarta artifacts, and to support the more widespread use of Jakarta.
• #13: This section describes how the transformer is being used within Open-Liberty builds.
• #14: The Eclipse Transformer is currently being used within Open Liberty builds, and has been used by the Open Liberty test organization. Within Open Liberty, the transformer has been used for several purposes. These are to update OSGi bundle archives, to enable existing Functional Acceptance Tests to run in Jakarta mode, and to convert certification tests to run in Jakarta mode. Complete transformations require updates in addition to package replacements. When updating OSGi metadata, package versions may also need to be updated.
• #15: Open liberty uses the transformer to enable running existing Functional Acceptance Tests in Jakarta mode. That is, repeating the existing tests with several test artifacts updated to their Jakarta equivalent. One of the test steps is to replace JavaEE features with the corresponding Jakarta features. This step is done outside of the transformer as a custom build step. Other test steps are to transform key test artifacts using the transformer. Test classes, test applications, server configurations used by the tests, are all transformed. Package references pervade several types of artifacts, and can occur in unexpected places. For example, JMS properties in activation specification elements of Open Liberty server configurations contain JMS class references, which imbed a JMS package reference, which must be transformed when running Open Liberty using Jakarta features.
• #16: The transformer was used to jump start certification testing: The transformer was used to transform JavaEE certification tests into Jakarta equivalents as an initial measure until the actual Jakarta certification tests were available.
• #17: Within Open Liberty, the Eclipse Transformer was added as an optional step for building OSGi bundle archives. The step was enabled by a new BND directive, “jakartaeeME”. Having added the directive to a project BND file, and having updated rules data with any necessary new package replacement data, the build automatically creates a second Jakarta version of the existing bundle archive for the project.
• #18: Here are several examples of the updates which are made to metadata. The first example is an update to a bundle manifest, which includes OSGi metadata. Note that updates are made to package names and to package version ranges.
• #19: This second example is a typical update made to a deployment descriptor. The example shows updates to resource references within a web module deployment descriptor.
• #20: This third example is an update to an Open Liberty server configuration, which requires an update to the properties of JMS activation specifications, which contain JMS class names. This transformation is performed when repeating Feature Acceptance Tests in Jakarta mode.
• #21: This next section provides a very brief overview of how the Eclipse Transformer works.
• #22: The transformer operates by recursively walking a target archive, directory, or file, copying and transforming all artifacts which are encountered.
• #23: Transformations are performed by an action framework. Action implementation classes are implicitly mapped to artifact types. For example, there is an action for java classes, an action for manifests, and an action for text files. The action framework selects a single action class for each artifact. A null action class is selected when no updates are to be made to a particular artifact. The action may update the name of the artifact and may update the contents of the artifact. The action finishes by summarizing what changes were made by the action, and the action framework collects and prints a report containing these summaries at the conclusion of processing.
• #24: The transformer may be invoked from the command line. This example shows the use of the JakartaTransformer helper class. The JakartaTransformer helper class uses prepackaged Jakarta rules property files to perform transformations. Alternatively, the Transformer class may be used directly, in which case rules property files must be specified using command line arguments.
• #25: Processing proceeds from inputs …
• #26: Through processing of rules data. In this example, default rules data, which is prepackaged with the transformer, is used.
• #27: (pause)
• #28: The target JAR file is transformed. Finally, a summary report is displayed. This examples shows that 57 artifacts were processed, of which, 6 were updated by the transformer.
• #29: (Pause)
• #30: The transformer is capable of updating several types of artifacts. Most prominent are java classes. Other notable types are manifests, service loader configuration files, and text-like files, including XML files. XML files include deployment descriptors and Open Liberty server configurations.
• #31: The transformer updates several types of archives. Archives are processed recursively. When transforming archives, the transformer is aware of the structure of the archive. Knowledge of archive structure is necessary for certain transformations, for example, transformations of service loader configuration files.
• #32: This section describes the specific updates which are made by the transformer.
• #33: The transformer performs several type of updates, in addition to updates to package references. When transforming a manifest, OSGi attributes which relate to bundle identity may be updated. That update is made to assign a new identity to the output bundle archive, which is necessary to distinguish the new, transformed, bundle archive from the original bundle archive. When transforming a manifest, OSGi attributes which contain package references may have new package version ranges assigned when updating the package references. Rules data may be specified to perform exact replacement of java string constants, independent and in addition to package updates. Text-like files may be updated using simple text replacement.
• #34: (pause)
• #35: (pause)
• #36: What updates are performed are configured through a set of property files. Most property files encode substitutions as initial-value – final-value pairs. Here are two examples. When invoking the transformer, what property files are used is specified using command line arguments, or are provided as pre-packaged resources, which is the case for invocations which use the helper class, JakartaTransformer. A set of Jakarta specific properties files is available within the transformer project. Jakarta specific property files may also be obtained from the Open Liberty GIT repository. Links to current Jakarta specific property files are provided at the end of this presentation.
• #37: Text substitutions are an important special case. Text substitutions use two tiers of property files. The first tier of property files map file patterns (which may be simple file names) to second tier property files. Second tier property files contain actual initial-final substitution pairs. The use of two tiers of property files enables the grouping of rule data which have common use, and enables the selection of rule data for use by specific files.
• #38: For example, within the prepackaged Jakarta rules data, a single property file was created for deployment descriptor updates. The several deployment descriptor file types were mapped by simple name to the common deployment descriptor rules property file.
• #39: For robustness, text substitutions have been specified using fully qualified class names. While verbose, this minimizes the likelihood of incorrect substitutions. There is at least one case of a Jakarta package replacement for which the sub-package is not replaced.
• #40: (Pause)
• #41: The development of the Eclipse Transformer, from inception through incubation, to release as an Eclipse hosted project, involved several IBM developers, and spanned much of 2020. That span saw the inception and proof of concept prototyping, saw an incubation period during which the prototype was extended and vetted through concrete use, and saw the eventual release of the transformer as a new Eclipse hosted open source project.
• #42: Inception was by Alasdair Nottingham and BJ Hargrave, who together noted the core technical problem; who conceived of the transformer as a way of quickly creating Jakarta artifacts from existing JavaEE artifacts; and who created the initial prototype transformer.
• #43: Subsequently, a team was formed of IBM Open Liberty developers, who used the transformer to transform Open Liberty artifacts. Web Profile features were selected as the first set of features which were to be transformed. The transformer was iteratively and quickly extended, using agile methodologies, to incorporate capabilities necessary to fully transform Web Profile artifacts. After the transformer was shown to successfully transform Web Profile features, the transformer was adopted as a part of the strategy for adding Jakarta support to the entirety of Open Liberty.
• #44: Finally, the transformer, which had been incubating in a private repository, was accepted as an Eclipse hosted open source project, and was migrated to a public repository.
• #45: (Pause)
• #46: The initial focus of the transformer was to provide a tool for application server development teams. The focus is expected to shift to usage by application developers. For example, to transform existing JavaEE applications, and to assist in assembling new Jakarta applications by combining existing JavaEE components with new Jakarta components.
• #47: Being rules driven, the transformer may be used to perform package replacements other than the JavaEE to Jakarta replacements which are provided by prepackaged properties files. One such application is to invert the JavaEE to Jakarta transformation. The embedded action framework may be extended to handle additional file types by adding new action implementations for new file types.
• #48: The current class action implementation performs simple substitutions. The implementation may be extended to perform additional types of updates, for example, code injection, or, non-string constant replacement.
• #49: (pause)
• #50: For more information about the Eclipse Transformer, or about the technologies used by the transformer, see these links.
• #53: In particular, two sets of rules data are available within public GIT repositories.
• #57: In conclusion, I would like to thank the many people who made the Eclipse Transformer possible, and for the audience for their patient participation.
• #58: (Pause)
• #59: The remaining time is allocated to questions and answers.