Running Oracle EBS

in the cloud

ANDREJS PROKOPJEVS
Lead Applications Database Consultant
About me

Apps DBA from Riga, Latvia.

Speaking SQL since 2001.

In Oracle world since 2004.
“In love” with Oracle EBS since 2006.

@aprokopjevs

Andrejs Prokopjevs [email protected]

Lead Applications Database Consultant
At Pythian since 2011 https://www.pythian.com/blog/author/prokopjevs/

© 2016 Pythian 2
ABOUT PYTHIAN

Pythian’s 400+ IT professionals

help companies adopt and
manage disruptive technologies
to better compete

© 2016 Pythian 3
TECHNICAL EXPERTISE Big Data: Harnessing the transformative
power of data on a massive scale

Advanced Analytics: Mining data for Databases: Ensuring databases

insights & business transformation are reliable, secure, available and
using data science continuously optimized

Cloud: Using the disruptive

nature of cloud for accelerated, Infrastructure: Transforming and
cost-effective growth managing the IT infrastructure
that supports the business

DevOps: Providing critical velocity

in software deployment by adopting
DevOps practices

© 2016 Pythian 4
EXPERIENCED GLOBAL EXPERTS

11,800 400 2
Systems currently Pythian experts Millennia of experience
managed by Pythian in 35 countries gathered and shared over
19 years

© 2016 Pythian 5
THE TOP 1.5% GLOBALLY
Technical
Tests
Sent: 4062
Passed: 562

2015
Resumes
Reviewed: Job Offers
12,711 Made: 189
Accepted:
174

Behavioral
Interviews
Conducted:
394

© 2016 Pythian 6
We are hiring !!!
Please visit:
https://www.pythian.com/careers/

HOT !!! Oracle Apps DBA

position in APAC region.

© 2016 Pythian 7
Agenda

• Certification basics.
• How to architect. Recommendations.
• Advanced configurations.
• R12.2.
• Microsoft Azure and Oracle Cloud review.
• Horizontal auto-scaling. Is this a supported configuration?

© 2016 Pythian 8
Certification basics

© 2016 Pythian 9
What is cloud?

• Google it - cloud computing is a type of Internet-based computing that provides

shared computer processing resources and data to computers and other devices on
demand.
• In other words, some virtualization platform service, which provides required resources on-
demand, on a shared hardware base.
• Hosting version 2? Not only that. Virtual hardware is not everything what you get today.

• IaaS
• SaaS
• PaaS

© 2016 Pythian 10
What is cloud?

© 2016 Pythian 11
What is cloud?

• Pros
• You do not need to think about your hardware, it's support, electricity, UPS or cooler in your
data center room, more. This is your cloud service provider responsibility.
• Hardware pool - sharing and distribution of the load.
• Transparency.
• "Pay per use" model.
• No termination fees.

• Cons
• Security?
▪ Design wise
▪ Encryption
▪ Single Point of Disaster (AWS console)
• IAM
• Root account
• Possible network latency with cloud service provider. © 2016 Pythian 12
What is cloud?

• Public cloud

• Private cloud

• Hybrid cloud

© 2016 Pythian 13
Certification

• SaaS or PaaS
• Ready solution. You don’t need to think about it.
• Example: Amazon RDS (DBaaS).

• IaaS
• Same rules apply if that would be your local physical
hardware. No changes.
• You are The Captain of your ship. ;)

© 2016 Pythian 14
Licensing
• Licensing Mantra
• http://www.oracle.com/us/corporate/pricing/cloud-licensing-070579.pdf
• AWS EC2, AWS S3, MS Azure.
• Customers are required to count each virtual core as equivalent to a physical core
▪ Database Standard Edition: <=4: vCore = processor licensed, >4: closest vCore multiple of 4 * 0.25 =
processors licensed
▪ Database Enterprise Edition: vCores * 0.5 = processors licensed (each virtual core is considered
equivalent to a physical core).
▪ Named User Plus
• Licensing models
• Pay per use rates, included in Cloud service provider pricing model (Amazon - only SE1 and
SE2)
• BYOL (Bring Your Own License)
• VMware – Sorry ! Nothing changed. You need to license the whole hardware pool.

• Cost efficiency: do your own calculations !

© 2016 Pythian 15
Oracle E-Business Suite

• Quick overview

• AWS EC2
▪ Amazon RDS for Oracle - NOT supported !
▪ Sorry, only IaaS.

• MS Azure
▪ Only IaaS.

• Oracle Cloud
▪ Something special is there. But we’ll talk about it a bit later.

© 2016 Pythian 16
 How to architect.
Recommendations.

© 2016 Pythian 17
Overview

• Our talk will be based on ”wrong cloud” Amazon EC2

• Oracle E-Business Suite R12.1

• Standard 1 Apps Tier & 1 DB Tier configuration

• High level overview

© 2016 Pythian 18
Important !

• You are the Solution Architect

• How you design it will be the way the system is going to run

© 2016 Pythian 19
#1: Network

• Everything should start with a proper Network design

• Amazon VPC
• http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html

• Region / Availability Zones

• Subnets
• Network interfaces
• Route tables
• Security
• Network ACLs
• Security Groups
• Internet Gateway
• Virtual Private Gateway
• IPSec VPN tunnel with your on-premise network
© 2016 Pythian 20
#1: Network

• IP address
• Private IP addresses – dynamic. Can’t be reserved and may change during instance restart.
• Public IP addresses – dynamic, but a static IP address can be assigned via Elastic IP feature.
▪ Note: Public IP traffic is routed through the public internet.

• DNS
• By default provided by Amazon
• Each region has it’s own sub-domain
• Automatically updated via internal DHCP
▪ But contains Private IP address in the name: ec2-10-10-10-1.eu-central-1.compute.amazonaws.com
• Route 53
▪ Configure your own DNS
▪ Assign names to instances, not IP addresses

© 2016 Pythian 21
#2: Instances

• Instance – actual virtual machine.

• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html

• Instance types
▪ General purpose
▪ Compute optimized
▪ Memory optimized
▪ Storage optimized
▪ Accelerated Computing

• It’s all about your “hardware” power

▪ vCores and Elastic Compute Units, T2 instances with CPU credits / burst
▪ Memory
▪ GPU
▪ more

© 2016 Pythian 22
#2: Instances

• Instance types #2
• On-Demand (default)
• Spot Instances
• Reserved Instances
• Dedicated Hosts
• Mainly affects pricing.

• AMIs (Amazon Machine Images)

• Image of the instance
• Public and Private repositories

© 2016 Pythian 23
#2: Instances
• Example that can be used for your Oracle E-Business Suite
• Apps Tier: m3.xlarge
▪ 4 vCores
▪ 13 ECUs
▪ 15 GB Memory
• DB Tier: r3.4xlarge
▪ 16 vCores
▪ 52 ECUs
▪ 122 GB Memory

• What AMI to use?

▪ EC2 comes with RHEL, SLES, Windows Server official package selection (pay per use licensing model)
▪ Oracle Linux available via Public AMI repository. Use for free, but don’t forget about ULN licensing terms.

• 122 GB RAM for EBS database?

▪ Sometimes memory size is not the primary pattern what instance type to select
▪ Each instance type has IOPS and throughput limits.
© 2016 Pythian 24
#3: Storage

• Documentation reference.
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Storage.html

• Storage types
▪ Amazon EBS (Elastic Block Store).
• Standard SAN-like disk volumes. Can be mounted to one EC2 instance at a time.
• SSD: gp2, io1 (PIOPS). HDD: st1, sc1.
• Different Throughput and IOPS limits.
▪ Amazon EC2 Instance Store (Ephemeral)
• Local disks.
• IMPORTANT: All data is lost once Instance is stopped or restarted.
▪ New! Amazon EFS (Elastic File System)
• NAS analog from Amazon – NFS.
• Note: Not available yet in all regions.
▪ Amazon S3
• By default used for storing AMIs and Amazon EBS disk snapshots.
© 2016 Pythian 25
#3: Storage
• Example that can be used for your Oracle E-Business Suite
• Apps Tier file system
▪ Elastic File System is Preferable, if scale-out plans are there.
▪ Elastic Block Store – standard option for Single-Tier approach.
▪ Make your own “NAS server” instance, mount Amazon EBS disks, and export them via NFS.
▪ gp2 standard type is absolutely enough. Based on IO credits / burst
• DB Tier
▪ Elastic Block Store
▪ Local File-System or ASM.
▪ No universal recommendation on IOPS / throughput. You MUST test, benchmark, and evaluate your own
system performance per requirements.
▪ Hint: Database Smart Flash Cache feature can greatly improve your performance. Configure it on Instance
Store (Ephemeral) disks.
SQL> show parameter db_flash_cache

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------
db_flash_cache_file string /dev/xvda1
db_flash_cache_size big integer 200G
SQL> © 2016 Pythian 26
#4: IOPS
• Most trickiest part to understand.

• IOPS / Throughput depends not only on Instance and

Volume type, but also on the size of the volume.
• Provisioned IO (io1) volumes are not always the efficient
option for IO intensive workload.
▪ Larger in size gp2 volumes can provide similar or better IO
performance than smaller io1.
▪ Less cost.
▪ Can experiment putting multiple gp2 into LVM drives.
• Wrong type chosen can introduce huge performance issues.
▪ Example: sc1

• gp2 / io1 based on 16KiB I/O size, st1 / sc1 based on 1 MiB

• Oracle Database: multiblock reads

▪ DB IOPS <> EC2 IOPS

© 2016 Pythian 27
#4: IOPS
• Classical reaction after long reading of the
IOPS docs in different areas (EC2, EBS,
RDS).

• Experiment
• Benchmark
• Find better setup that will suit your needs

© 2016 Pythian 28
#5: Backup / Recovery

• Documentation reference.
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html

• EC2 provides EBS snapshot feature only

▪ Apps tier: Do snapshots for Apps Base file system.
▪ DB tier: Setup RMAN on dedicated volumes. Do periodic snapshots. Or NAS.
▪ Can be automated via CLI.

© 2016 Pythian 29
#6: Monitoring

• CloudWatch.
• https://aws.amazon.com/cloudwatch/

• Free:
• Very limited.
• Not-free:
• Complete monitoring solution for the instances.
• Alarms.
• SWS / SNS integration.

• But not very versed for Oracle.

• You can use your own custom monitoring.
• Setup EM agents.

© 2016 Pythian 30
#7: Amazon RDS

• You can still use Amazon RDS for Oracle for other components.
• Oracle Fusion Middleware 12c
• Not supported with Oracle Fusion Middleware 11g, but with some “tweaks” you can still get
your Metadata Repository loaded into RDS instance.
▪ Main issue: no SYS as SYSDBA access. Replaced with ORACLE_MASTER and RDS PL/SQL API.

• Example:
• Oracle Access Manager and Oracle Internet Directory for Single Sign-On.

© 2016 Pythian 31
 Advanced
configurations

© 2016 Pythian 32
#1: Oracle RAC

• Is NOT natively supported *.

• No shared storage.
• No multicast support for interconnect.

• But… non-official solutions.

• Deploying Scalable Oracle RAC on Amazon EC2
▪ https://aws.amazon.com/articles/7455908317389540
▪ * - became official guide by Amazon.
• Flashgrid VSAN solution.
• Do your own interconnect via OpenVPN, try to adopt Amazon EFS (NFS) via direct NFS.

© 2016 Pythian 33
#2: Apps Multi-Tier / PCP

• No limits.
• Shared file-system can be implemented via EFS or own NFS solution.
• Or use non-shared file system approach using separate Amazon EBS volumes.

• Elastic Load Balancing

• https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/what-is-load-
balancing.html?icmpid=docs_elbv2_console

• Has all requirements to front-end Oracle E-Business Suite

▪ Sticky session
▪ SSL
▪ more

© 2016 Pythian 34
#3: External Tier

• Same as standard Apps Multi-Tier.

• DMZ network restrictions can be implemented via a dedicated VPC security group.

© 2016 Pythian 35
#4: SSL

• Native SSL can be implemented in the same way as on-premise.

• AWS Certificate Manager is available to maintain the certificates.
• Can be just front-ended to Elastic Load Balancer with SSL termination.

© 2016 Pythian 36
#5: Integrations

• Evaluate your network latency having Hybrid cloud.

• For heavy data exchange systems integrated it might be good to move them to your
Oracle E-Business Suite on the same side.

• Example:
• Data Warehouse
• ETL

• Or adjust your expectations and SLA.

© 2016 Pythian 37
#6: Disaster Recovery

• Availability zones.
• Regions.

• IaaS – similar “on-premise” approach.

• For initial clone, Amazon EBS volume snapshots are transferable.

• Amazon RDS for Oracle – built-in feature (DBaaS).

© 2016 Pythian 38
#7: Encryption

• Implement TDE.
• Amazon RDS for Oracle (not for Oracle Ebiz) also supports that.

• Or implement encryption of Amazon EBS volumes on the storage layer.

© 2016 Pythian 39
R12.2

© 2016 Pythian 40
What is different with R12.2?

• Everything that is related to R12.1.

• ADOP
• Private IP addresses
▪ Number of FND tables must contain valid IP addresses of all nodes.
▪ EC2 Instance restart changes the private IP – it must be updated.
• Validation of the /etc/hosts
▪ Same private IP addresses – hosts file has to enlist them, and with required format.
• IO throughput planning is important
▪ Heavy Apps run / patch file system synchronization through online cycles.
▪ DB workload handling edition objects.
▪ Example: “fs_clone force=yes” run duration on 220 GB standard gp2 volume: 42 minutes.

• You can setup a custom OS service

• Run during instance startup to update hosts file and FND table with $(hostname -I) value.
• Any other node dependencies via AWS CLI.
© 2016 Pythian 41
Microsoft Azure

© 2016 Pythian 42
Microsoft Azure

• Same IaaS.
• In respect to Oracle E-Business Suite it has all main analogs available.
• Virtual Network
• Linux Virtual Machine support
• Load Balancing
• Storage
• Site Recovery
• more

• Important note: Compute limits.

• Main disadvantage comparing EC2 and Azure was the compute option availability. EC2
provided more powerful capacity options. Almost 10 times.
• Things got changed a bit.
▪ 20 vCore max quota (30 West Europe region). But can be extended via support request.
▪ 100+ GB RAM
▪ Enough storage options © 2016 Pythian 43
Oracle Public Cloud

© 2016 Pythian 44
Oracle Public Cloud

• Announced and released few months back.

• Same IaaS approach.
• But… there are additional PaaS options. Old picture. 12.2.6
• Database Cloud Service images are available
• Exadata Cloud Service now too.
• Easy multi-node provisioning for Oracle EBS.
• “Lift and Shift”
• One-button your on-premise instance migration to
cloud.
• Marketing. J
• Quick deployment for rapid development
needs (provisioning).

• Completely based on Oracle’s own stack.

© 2016 Pythian 45
Oracle Public Cloud - Integrations

• As we know, Oracle EBS is really bound to

Oracle’s own products in terms of integration
requirements.
• With other cloud service providers it’s IaaS and
install / patch / maintain everything yourself.

• Welcome to Integrations Cloud Service.

© 2016 Pythian 46
That’s it about Oracle Public Cloud

• It’s still too early to tell anything interesting with technically proven stuff.
• No real implementation production experience.
• Information is still being released and published.

• Follow:
• https://www.facebook.com/groups/EBS.SysAdmin/
• https://twitter.com/Oracle_EBS
• Getting Started with Oracle E-Business Suite on Oracle Cloud (Doc ID 2066260.1)

• Recommendation: Do not rush!

• Implement
• Test
• Evaluate
• Migrate
© 2016 Pythian 47
Horizontal auto-scaling

© 2016 Pythian 48
What is scaling?

• Vertical scaling
• Increase of the compute resources
▪ Change instance type with more RAM, more
CPU power
• Might dramatically increase your cost
• Still has limits. J

• In respect to Oracle EBS

• Increase your JVM memory heap size
• Increase your JVM process count
• R12.2: Add more oacore_serverX WLS
servers

© 2016 Pythian 49
What is scaling?

• Horizontal scaling
• Increase of the compute resources
▪ Add more nodes
• Flexibility

• In respect to Oracle EBS

• Add more apps tiers.

© 2016 Pythian 50
Challenge

• Oracle E-Business Suite is very depended on it’s internal configuration repository

(FND_NODES)
• Nightmare with R12.2 and ADOP.

• Vertical scaling
• No issues. Just go and tune the instance.
• Horizontal scaling
• Manual effort with “perl adclonectx.pl addnode”.
▪ Find the best combination of pairs file to minimize post manual correction of the context file and instance
configuration.

• What if I need to delete the node?

• Downtime, setup_clean, re-run of all autoconfigs on each node in 2 rounds.
• R12.2 additional: patch fs context file re-upload to the database.
© 2016 Pythian 51
What if we want to automate it?

• Business case:
• As for a simple web-server we would like to automatically add web nodes on-demand, if
existing stack is overloaded.

• EC2 Auto Scaling Documentation

• https://aws.amazon.com/documentation/autoscaling/

• Auto-Scaling group is created

• Minimum, maximum, and desired node count parameters
• Scaling plans – when and how to scale. Specific time, or can be based on CloudWatch
monitoring. Rules for scaling in.
• Launch configurations: Instance configuration.
▪ Create your private AMI from existing Apps tier.
• Automatically adding your new node created into the Elastic Load Balancer configuration.
© 2016 Pythian 52
What if we want to automate it?

• What can happen when we start a new node created from AMI?
• We can put a custom first-boot shell script which will handle certain actions.
• cloud-init standard method.
▪ By default, user data and cloud-init directives only run during the first boot cycle when you launch an
instance.

• Pricing for Auto-Scaling

• It is free.
• You pay only standard rates running your additional EC2 instances.

• Limits

© 2016 Pythian 53
Adding a new Web Node

• OS setup
• Software, kernel, limit configuration is coming from created private AMI.
• Hostname.
▪ We need to generate a unique hostname, set it on the server, DNS, update /etc/hosts (R12.2).
• Mount the Apps Base storage layer.
• Apps setup
• Generate required pairs file.
• perl adclonectx.pl addnode
• Run AutoConfig on all existing nodes. Or, at least, “generatetns” step.
• Optional: if sqlnet.ora tcp.invited_nodes restrictions are enabled, “lsnrctl reload
LISTENER_NAME” has to be issued on all existing nodes.
• Optional: any custom post configuration
• Service restart on new created node.
• Elastic Load Balancer – automatically.
• Will enable the node once the TCP socket ping succeeds.
© 2016 Pythian 54
Removing a Web Node

• Instance termination
• Elastic Load Balancer will drop the removed instance from it’s configuration
automatically.
• Apps setup
• Master node to trigger the node deletion process.
• R12.1
▪ No process of node deletion.
▪ Concurrent Service Managers configured for new node can be just disabled from back-end.
▪ OAM dashboard – sorry, you can’t avoid the red status there.
▪ File system cleanup.
• R12.2
▪ Abandon the node.
▪ perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-node
▪ Optional: “txkSetAppsConf.pl -configoption=removeMS” on existing nodes if OHS is pointed to all
WebLogic nodes in the cluster, including the removed one.
▪ File system cleanup.
© 2016 Pythian 55
Re-use of a removed Web Node

• What if we do not want to generate new unique node names every time, and just
reuse the same integers available?

• R12.1
• Easy. But we should not delete the INST_TOP then.
• Re-enable the Concurrent Service Managers.
• Just launch the instance, configure the OS part, and restart the services.

• R12.2
• Abandoned node – only a complete re-add. Even if we do not delete the node and do not
delete the file system content.

© 2016 Pythian 56
Periodic housekeeping

• Once you have a downtime window, it’s worth to go through “setup_clean” process
to have everything re-registered clean and drop garbage.

© 2016 Pythian 57
R12.2 considerations

• Abandoned node / Delete node process introduction is useful.

• But what happens if we scale-out or scale-in during an open ADOP online patch
cycle?
• It worked during a test case. But I wouldn’t give a 100% guarantee that all is going to be good
during the next cutover. I was lucky.
• Scale-in process is Abandon process. It is handled during the cycle correctly.

• And what can happen if we do all these our activities during a real phase execution,
like cutover?
• Not tested, but expect everything will break in a quite major manner.
• CloudWatch doesn’t know anything about what’s going on in Oracle EBS.
• Maybe Auto-Scale group should be disabled during these maintenance activities.

© 2016 Pythian 58
Summary

• Oracle EBS implementation on cloud is more less straightforward.

• Mostly IaaS. Oracle Public Cloud offers SaaS / PaaS for Oracle Database and
other integrations, like SOA.
• IO performance is the main bottleneck risk. Good design, evaluation of estimates
and testing are very important.
• Security is important.
• Cost efficiency is a subject of detailed calculation.
• Auto-scaling - interesting possibilities exist, but not supported yet.

© 2016 Pythian 59
THANK YOU
Q&A

© 2016 Pythian 60