Dynamic Groovy Edges

1
Managing Change and Achieving
Regression Isolation
Dynamic Groovy Edges
May 21st, 2012
Prepared for:
Triangle Java User’s Group (TriJUG)

2
Who Am I? Well…
 Java Architect with ICF Ironworks
 Adjunct Professor
 Started with HTML and Lotus Notes in 1992
• In the interim there was C, C++, VB, Lotus Script, PERL, LabVIEW,
etc.
 Not so much an Early Adopter as much as a Fast Follower of Java
Technologies
• Learned Java 1.1 in 1997, J2EE in 1999
 Alphabet Soup (MCSE, ICAAD, ICASA, SCJP, SCJD, PMP, CSM)
 LinkedIn: http://www.linkedin.com/in/iamjimmyray
 Blog: http://jimmyraywv.blogspot.com/ Avoiding Tech-sand

3
Managing Change and Achieving
Regression Isolation
Dynamic Groovy Edges

4
JEE Application Architecture
 Arranged into layers
 When there are N layers:
• There are N-1 layer junctions
• There are (N-1) X 2 layer edges
In this diagram there are three
layers, two layer junctions, and four
layer edges.
We
control
edges

5
The Issue with Layers
 Often written by someone else
• You often have little control or influence over the implementation of the
layer to which you must integrate
• You control your side (edge) of the interface junction
 Notorious change zones
• Changes occur at different paces

6
Keeping Up the Pace
 Pace Layering Approach to Application Design
• http://www.doozer.com/developer-exchange/entry/accelerating-innovation-
by-adopting-a-pace-layered-application-strategy
Complex systems can be decomposed into multiple layers, where the layers change at different
rates. The “fast layers” learn, absorb shocks and get attention; the “slow layers” remember,
constrain and have power. One of the implications of this model is that information architects can
do what they have always done–slow, deep, rich work; while tagging can spin madly on the
surface. If it is worth keeping, it will seep down into the lower layers. – Donna Maurer

7
When Layers Change
 In Java web applications, regression analysis and testing are
usually tied to the boundaries of the application or deployment
artifacts: JARs, WARs, EARS etc.
• Upstream and downstream dependencies also come into play
• Not to be confused with functional testing of changed components
 If you redeploy the entire WAR or EAR, you could find yourself
testing that entire application boundary.
• As well as its integration junctions (critical patches not withstanding)

8
Detecting Changes
 How do we detect changes to web services as soon as possible?
• XMLUnit/JUnit Tests
• Easily written in Groovy
• Hudson scheduled Ant tasks
DIFFERENCE CODE:19:CHILD_NODELIST_LENGTH_ID:Comparing 2 nodes with different
numbers of children
Control(prod) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element minOccurs="0" name="address" type="tns:address"/>
<xs:element minOccurs="0" name="employeeId" type="xs:string"/>
<xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/>
</xs:sequence>
Target(dev) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element minOccurs="0" name="address" type="tns:address"/>
<xs:element minOccurs="0" name="department" type="tns:department"/>
<xs:element minOccurs="0" name="employeeId" type="xs:string"/>
<xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/>
</xs:sequence>

9
Demo – Detecting Changes
 Groovy Console
 JUnit
 XMLUnit
 Apache 2.2
 Tomcat 6.0.35

10
CI Using Groovy XmlUnit Tests
 Wrap Groovy script with Ant
 Execute in Hudson build
 Use Gradle with the Groovy Plugin
• Run your Groovy test within Gradle build

11
Managing Change
 How do we better manage the change between the layers?
• Understand that we cannot stop change
• Reduce regression testing needs by shrinking regression boundaries
• Prepare our teams for the changes with more dynamic reaction
techniques
• Start with Groovy beans
• Or…Start with Java beans and move to Groovy as needed
• Detect changes as early as possible, and before they impact
downstream systems

12
Groovy Edges
 Build your edges with Groovy (or other dynamic
JVM language) objects
• Groovy can be written as a script and parsed at run-
time and compiled into Java byte-code and then
loaded by the Groovy class loader.
• Groovy scripts can contain Groovy and Java syntax
in the script file (*.groovy).
• Groovy can use the same Java APIs that are used
by traditional Java code.
• Groovy adds features that make it easier to work
with XML (primary language of choice for most
integrations)
• Markup Builder
• XML Slurper

13
Groovy Markup Builder
 Allows developers to use “builder patterns” to construct XML
def writer = new StringWriter();
def xml = new MarkupBuilder(writer);
xml.'ser:writeEmployee'('xmlns:ser':"http://service.rjug.ironworks.com/"){
employee{
dateOfBirth(emsRequest.getEmployee().getDateOfBirth())
firstName(emsRequest.getEmployee().getFirstName())
lastName(emsRequest.getEmployee().getLastName())
middleName(emsRequest.getEmployee().getMiddleName())
employeeId(emsRequest.getEmployee().getEmployeeId())
hireDate(emsRequest.getEmployee().getHireDate())
address{
addressLine1(emsRequest.getEmployee().getAddress().getAddressLine1())
…

14
A Word About Hard-Coding (or not)
 When is hard-coding not really?
• Our internal beliefs have shifted slightly.
 How about when it is in Groovy?
• Especially when that Groovy is treated as content and not part of a
rigid deployment architecture?
• Still have to work within the boundaries (constraints) of the JVM

15
Groovy XML Slurper
 Groovy construct for efficient partial XML processing.
EmployeeManagerServiceResponse emsResponse = new EmployeeManagerServiceResponse();
GPathResult response;
try{
response = new XmlSlurper().parseText(rawResponse.getWriter().toString());
emsResponse.getEmployee().setFirstName(response.'return'.firstName.text());
emsResponse.getEmployee().setLastName(response.'return'.lastName.text());
…

16
Groovy XML Slurper – Another Example
slurper.jobList.job.each {
it.docs.doc.each {
doc = new DocDto();
docs.add(doc);
doc.setDocumentId(new BigInteger(it.documentId.toString()));
//doc.setDcmntNbr(new BigInteger(it.dcmntNbr.toString()));
doc.setOriginalFilename(it.originalFilename.toString());
doc.setPhasecode(it.phasecode.toString());
doc.setRcvdDt(it.rcvdDt.toString());
doc.setStatuscode(it.statuscode.toString());
doc.setStepcode(it.stepcode.toString()); …
… <jobList>
<job>
<docs>
<doc>
<originalFilename>562132403073245.pdf</originalFilename>
<documentId>42</documentId>
<rcvdDt>2009-01-10 00:00:00.0</rcvdDt>
<phasecode>A</phasecode>
<stepcode>B</stepcode>
<statuscode>C</statuscode>
</doc>
</docs>…

17
A Word About Groovy Slurping XML Payloads
 No longer tied to JAXB
• This is uncomfortable for some traditional Java developers
 Can use Groovy closures on collections (even nested collections)
 Easy to enable/disable getter calls, especially if Groovy is stored
outside of rigid deployment structure (CMS, Apache, etc.)
• This is scary to some
• Still requires Configuration Management rigor
• It’s about risk tolerance
• It could be that this approach is better suited for non-production
environments within certain organizations

18
Groovy XML Slurper Trade-offs
 Use the Groovy XML Slurper when you do not have to parse the
entire XML
• Slurper uses paths expressions as apposed to constructing a DOM
• Path expression might be executed multiple times
• Could be slower in larger, more complex XML
 There are also differences when editing XML
• The XML Parser allows for edits inline
• The XML Slurper would require another “slurp” after changes are
applied.
 The XML Slurper is well suited for simple web service clients
• Allows direct access via getters to “return” object graph

19
Groovy Edge Component – Web Service Client
 For SOA interfaces implemented with web services, web service
clients are primarily the target edge component for dynamic
updating.
 Using Groovy to write these web services clients makes it easier to
update clients.
• Features can be added or removed and quickly “redeployed” via
content changes, and not necessarily Java deployments
 Spring will reload the Groovy web service client when it changes
 This provides a great solution for prototyping in an Agile
environment.
 For MVC containers, the controller is the edge component that is
up for “Groovy-enizing”, “Groovy-ization”, etc.

20
Spring Beans
<lang:groovy id="GroovyInterfaceEdge" script-source="${groovy.url}"
refresh-check-delay="1000" >
<lang:property name="key" value="value"/>
</lang:groovy>

21
Groovy Location Property
groovy.url=http://localhost/groovy/EmployeeManagerServiceImpl.gro
ovy
 Not only that, but…
• Store your properties outside of the WAR/EAR, reachable via URL

22
Loading Properties

<bean id="services.propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlacehold
erConfigurer">
<property name="location">
<value>http://localhost/properties/service.properties</value>
</property>
</bean>

23
Demo – Web Services Provider
 Simple JAX-WS Bottom-up web service
 Environment:
• Apache2.2
• Tomcat 6.0.35
• Java 1.6 JRE
• JAX-WS RI 2.2
• JAXB 2.2.3
• SoapUI 4.0.0
• Spring Data Layer – MongoDB Data Repository
• MongoDB NoSQL Database (Upcoming TriJUG Event: 07/16/12 Deep
Mistry Mongo DB)

24
Demo – Web Services Consumer
 Simple Spring WS Client
 Environment:
• Apache2.2
• Tomcat 6.0.35
• Java 1.6 JRE
• Spring Framework 3.0.x
• Spring WS
• Groovy Web Services Framework

25
This Is Not Bleeding Edge
 Yet, it is still new to a lot of Spring users
 January 2009: http://www.ibm.com/developerworks/java/library/j-
groovierspring2/index.html
 At the NVSS NFJS conference in Reston in April 2012, there were
still sessions on Spring and Groovy integration with dynamic beans
• NFJS Due in Raleigh, NC – August 24-26
• http://www.nofluffjuststuff.com/home/main

26
Spring Bean Configuration Issues
 Spring needs to be able to locate Groovy script content in order to
load configurations.
ERROR - ContextLoader.initWebApplicationContext(220) | Context
initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'employeeManagerService' defined in file
[C:Toolsapache-tomcat-6.0.16w
ebappsDynamicGroovyTutorialWEB-INFclassescontextservice_ws.xml]:
Cannot resolve reference to bean 'employeeManagerServiceImpl' while
setting bean property
'groovyImpl';

27
Spring Bean Config With Content Proxy
 Use a Content Proxy
• The Content Proxy is deployed into a separate WAR and addressable
via a URL (perhaps a RESTful web service) that would retrieve the
dynamic Spring Bean configuration scripts
• If the scripts are not found in script repository, the Content Proxy can load
the scripts from a backup site or even from inside the application boundaries
– Default Location.
• The Content Proxy is a layer of indirection to better manage the link
between your Spring application and the content management system that
you may be using.
• Content Proxies are also very helpful when scripts (or other content) are
loaded from a CMS through an API that would not link directly to the Spring
Bean configurator.
• Provides a layer of indirection and intelligence for Cache Forward
Architectures.

28
Avoiding the Race Condition
 Using a ServletContextListener allows you to specify how long the
application will try to seek the ContentProxy (or other URLs) in
other WARs that still may be spinning-up, before it lets the rest of
the application load.
• Reduces the “race-condition” introduced by URLs across multiple
WARs

29
ServletContextListener web.xml Configuration
…
<context-param>
<description></description>
<param-name>proxyUrl</param-name>
<param-value>http://localhost/proxy.html</param-value>
</context-param>
<context-param>
<description></description>
<param-name>stopInterval</param-name>
<param-value>60000</param-value>
</context-param>
<listener>
<listener-class>com.ironworks.rjug.container.listener.ContentProxyListener</listener-
class>
</listener>
…

30
Inline Groovy – You can, but why would you?
package com.ironworks.framework.scripting;
import org.springframework.core.io.ResourceLoader;
import org.springframework.scripting.ScriptSource;
import org.springframework.scripting.support.*;
public class URLScriptFactoryPostProcessor extends ScriptFactoryPostProcessor{
@Override
protected ScriptSource convertToScriptSource(String beanName,
String scriptSourceLocator, ResourceLoader resourceLoader) {
if (scriptSourceLocator.startsWith(INLINE_SCRIPT_PREFIX)) {
return new StaticScriptSource(
scriptSourceLocator.substring(INLINE_SCRIPT_PREFIX.length()), beanName);
}
else if (scriptSourceLocator.startsWith("classpath")) {
return new
ResourceScriptSource(resourceLoader.getResource(scriptSourceLocator));
}
else{
return new URLScriptSource(resourceLoader.getResource(scriptSourceLocator));
}
}
}

31
Spring Bean Delegating Construct
 Use a Spring Bean Delegate construct as an alternative to
ServletContextListener and ContentProxy for handling potentially
unreliable or dynamic URLs during application startup
 Deploy JAXB version of services, but leave them open to
replacement by Groovy, without needing another deployment
• Still able to use Dynamic Groovy Edges, if JAXB version of service
client requires updating
<bean id="iw_service.getData" class="com.ironworks.vdc.framework.service.DelegatingService">
<property name="delegates">
<list>
<ref bean="iw_service.getDataGroovy" />
<ref bean="iw_service.getDataJaxb" />
</list>
</property>
</bean>

32
Spring Bean Delegate Construct Code
public final class DelegatingService<REQUEST extends ServiceRequest, RESPONSE extends
ServiceResponse>
implements Service<REQUEST, RESPONSE> {
private List<Service<REQUEST, RESPONSE>> delegates =
new ArrayList<Service<REQUEST, RESPONSE>>();
@Override
public RESPONSE execute(final REQUEST request) {
for (Service<REQUEST, RESPONSE> service : delegates) {
if (!(service instanceof AvailabilityStatus)
|| ((AvailabilityStatus) service).isAvailable()) {
return service.execute(request);
}
} throw new UnsupportedOperationException(
"No service implementation available to execute request");
}
public void setDelegates(final List<Service<REQUEST, RESPONSE>> delegates) {
this.delegates = delegates;
}
}

33
AvailabilityStatus Interface
public interface AvailabilityStatus {
/**
* Indicate whether this service instance is available.
*
* @return true if the service is available, false otherwise.
*/
boolean isAvailable();
}

34
Spring Delegate Construct Code Explanation
 DelegatingService is substituted into the class attribute of the bean
definition
 Possible “delegates” are added to the property list as objects
 DelegatingService looks for viable services in the list of “delegates”
• Could be services that implement the AvailabilityStatus interface, and
are available (isAvailable())
• Beans that implement AvailabilityStatus, but are not available our by-passed
– This allows Groovy substitutes to be enabled/disabled quickly
• Could be services that do not implement AvailabilityStatus

35
Some Tradeoffs for Groovy Approach
 Groovy Web Clients
• Pros
• Easier to program with Getters and Setters approach to accessing and
mutating XML objects for request and response
• Markup Builder
• XML Slurper
• Cons
• Lose auto-generation of object graphs via JAXB
• Lose tighter integration to WSDL/XSD and makes it harder to track down
changes.
• It’s not for every change…more complex WSDLs, possibly with nested
object architectures, would not be good candidates for Groovy re-write

36
What About All Those Classes?
 Almost every technical decision comes with a trade off…
 When using dynamic Groovy with the approach we have outlined,
obviously we are generating more “non-setup”, Groovy classes in
the JVM.
• Every time Spring detects a changed script, Groovy will parse it and
load another class.
• Depending on your JVM, GC settings and JVM memory settings, you
could experience OOM issues with PermGen

37
Does Your JVM Have a PermGen?
 IBM: No PermGen, classes exist in the native C-Heap
 Oracle/WebLogic JRockit: No PermGen, classes exist in the
native C-Heap
 Oracle/Sun Hotspot: Yes, PermGen in Java Heap
• Prior to Java 5, the Hotspot JVM assumed that classes were forever
• Then came CMS (Concurrent Mark Sweep) class unloading settings

38
What Can We Do With PermGen?
 Use GC settings for the CMS Collector
• Java 1.6: -XX:+CMSClassUnloadingEnabled
• Java 1.5: -XX:+CMSPermGenSweepingEnabled
 Both require CMS GC enabled
• This may not be suitable for all applications
 Increase size (not really desirable)
• Switch to 64 bit, if you have not already
 Increase JVM bounce frequency (not really desirable)
• Not really a good long term strategy

39
Don’t Use PermGen.
 Some of our projects use JRockit
• While we do not see the same sensitivity to PermGen in our dynamic
environment, occasionally we still see issues with class memory areas
• Mostly in non-production environments where we tend to be more
dynamic
 Problem: JRockit does not exist beyond Java 7
• Oracle has announced that it is being merged with Hotspot in Java 7

40
Use Java 7?
 In Java 7, PermGen is supposed to go away over time.
• This implies better class unloading and memory management
• JRockit merges with Hotspot
• When?
• http://javaeesupportpatterns.blogspot.com/2011/10/java-7-features-
permgen-removal.html
• http://blogs.oracle.com/java/entry/java_7_questions_answers

41
Groovy Spring MVC Controllers
 Spring MVC controllers are another example of edge components
that could benefit from more dynamic deployment and less rigid
deployment structures
• Remember: Just because it is marked as Groovy (*.groovy) does not
preclude you from writing some Java
<lang:groovy id="claimsController" script-source="${groovyClaimsController}" refresh-check-
delay="1000" >
<lang:property name="view" value="claim_summary"/>
<lang:property name="detailView" value="claim_detail"/>
<lang:property name="unsolicitedView" value="claim_unsolicited"/>
<lang:property name="errorView" value="claim_error"/>
<lang:property name="serviceDownView" value="claim_service_down"/>
</lang:groovy>>

42
Development Environment Configuration
 Eclipse Helios SR2
 Groovy-Eclipse plugin (The Codehaus) – 2.5.1.xx-2011027…
 Groovy 1.8.6
 Maven 2.0 (m2) Eclipse Integration – Sonatype, Inc
• gmaven-plugin – 1.4.0
 MongoDB 2.0.4

43
Summary
 We cannot control changes in layers beyond our influence or
control
 We can better prepare for change by using dynamic technologies
in the edges of interface junctions.
• Groovy blurs the once clear line between hard coding and properties
configurations.
• By using Spring to dynamically load Groovy scripts, stored outside of
web applications, web applications can narrow regression boundaries
by reducing the need to deploy EARs and WARs.
• The Content Proxy provides a layer of indirection that smoothes the
process for locating dynamic scripts an providing digestible data for the
Spring Bean Configuration process.
• Testing WSDL and XSD artifacts with XMLUnit provides early warning
of web service changes and failures

44
Special Thanks To:
 Laith Al-Samir – ICF Ironworks
• Spring WS application and Groovy WS Client
 Mike Killeen – ICF Ironworks
• Groovy/XMLUnit/JUnit WSDL/XSD tests
 Vince Jorrand – ICF Ironworks
• Spring Bean Delegates Construct

Dynamic Groovy Edges

Recommended

More Related Content

What's hot (20)

Similar to Dynamic Groovy Edges (20)

Recently uploaded (20)

Dynamic Groovy Edges