Oracle ORAchk & EXAchk overview

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle ORAchk & EXAchk
Oracle Stack Health Checks
Version 18.2.0

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Agenda
3
Overview
Autonomous Usage
Customizing Execution & Output
Oracle Health Check Collection Manager
Integrating with Oracle Enterprise Manager Cloud Control
Integrating with 3rd Party Tools
Troubleshooting
ORAchk Specific Features
EXAchk Specific Features
1
2
3
4
5
6
7
8
9

Agenda
4
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Automatic proactive warning
of problems before they
impact you
5
Get scheduled health reports
sent to you in email
Why Oracle ORAchk & EXAchk
Health checks for most impactful
reoccurring problems
Runs in your environment
with no need to send
anything to Oracle
Findings can be integrated
into other tools of choiceEngineered
Systems
Non
Engineered
Systems
EXAchk
Common Framework
ORAchk

• Release cycles are targeted to 90 days, with Beta made available
approximately two weeks before production
o Interim releases may be released sooner if significant new checks are
added before scheduled release
o Also timed to coincide with major software releases and / or hardware
changes
• Maintains backward compatibility
• Continuous improvement from:
o Enhancement requests / feedback from customers
o Exadata MAA Best Practices team
o Critical Issues discovery
o Internal testing, proofs of concepts & Support partners
6
Development Methodology

Engineered Systems
Oracle Exadata Database Machine
Oracle SuperCluster
Oracle Private Cloud Appliance
Oracle Database Appliance
Oracle Big Data Appliance
Oracle Exalogic Elastic Cloud
Oracle Exalytics In-Memory
Machine
Oracle Zero Data Loss Recovery
Appliance
Oracle ZFS Storage Appliance
Systems
Oracle Solaris
Cross stack checks
Solaris Cluster
OVN
ASR
7
Oracle Stack Coverage
Oracle Database
Autonomous Database
Standalone Database
Grid Infrastructure & RAC
Maximum Availability Architecture
(MAA) Scorecard
Upgrade Readiness Validation
Golden Gate
Application Continuity
Enterprise Manager Cloud
Control
Repository
Agent
OMS
Middleware
Oracle Identify and Access
Management Suite (Oracle IAM)
E-Business Suite
Oracle Payables
Oracle Workflow
Oracle Purchasing
Oracle Order Management
Oracle Process Manufacturing
Oracle Receivables
Oracle Fixed Assets
Oracle HCM
Oracle CRM
Oracle Project Billing
Siebel
Database best practices
PeopleSoft
Database best practices
SAP
EXAdata best practices

ORAchk / EXAchk Collection Manager Enterprise Manager ELK Stack
8
Architecture Options
Health
Checks
Run
Checks
HTML
Results
Email
Notification
Oracle
Database
Many Instances One Instance One Instance One Instance
Oracle
Database
Elastic
Search
SQL
Results
XML
Results
JSON
Results
View enterprise wide
results via Collection
Manager interface
results via Enterprise
Manager interface
results via Kibana
dashboards

ORAchk
Operating System Versions Supported
Linux / zLinux Oracle Linux/RedHat 4,
5, 6, 7
SuSE 9,10, 11, 12
Oracle Solaris SPARC / x86-64 9, 10, 11
AIX 5.2, 5.3, 6.1, 7.1, 7.2
HPUX Itanium / PA-RISC 11.23, 11.31
Microsoft Windows 2008, 2012
EXAchk
• All Oracle Engineered Systems
– Apart from ODA, which uses ORAchk
9
Supported Environments

If Oracle Clusterware is installed
1. Download latest version
– ORAchk : 1268927.2
– EXAchk : 1070954.1
2. Copy the zip file to the installation
system and extract:
– EXAchk:
in /opt/oracle.SupportTools/exachk as the
Oracle Grid Infrastructure home owner
– ORAchk:
in CRS_HOME/suptools/orachk as the
Oracle Grid Infrastructure home owner
If Oracle Clusterware is not installed
1. Download latest version
– ORAchk : 1268927.2
– EXAchk : 1070954.1
2. Copy the zip file to the installation
system and extract:
– EXAchk:
in /opt/oracle.SupportTools/exachk as the
root user
– ORAchk:
in a convenient location as Oracle software
install user or Oracle Database home owner
10
Installation

• Run as root (recommended)
o ORAchk/EXAchk will su to lower privileged
owners of RDBMS or grid homes
o To specify a user other that root for these
situations:
• Run as RDBMS or GRID Home Owner
o User must be able to switch to root for root level
checks – several options:
1. Provide the root userid password at prompts
or
2. Set up sudo
or
3. Pre-configure passwordless SSH connectivity
or
4. Allow ORAchk/EXAchk to configure private keys for
remote nodes
Which User to Run as
Connect via
SSH &
Run Checks on
Default User
Change User By
exporting user id in this
Environment Variable
Exadata Storage
Server
root RAT_CELL_SSH_USER
InfiniBand
switches
root
(when run as root)
RAT_IBSWITCH_USERnm2user
(when run as other user)
11
Note: You may only choose from the provided lower privileged account
Note:
•On SuperCluster you can use Role Based Access Control (RBAC) to
execute root privileged checks, no root user required.
•root checks must be run as a user with a root equivalent access role
•On Exalogic it is only supported to run as root

Remote node connection without passwordless SSH
• ORAchk/EXAchk will:
1. Prompt for remote node password
2. Login to remote node and generate private and public key pair on remote node
3. Copy contents public key into the .ssh/authorized_keys file of remote node and delete the public key
from remote node
4. Copy private key of remote node into local node and use as identity file to make future connections
• Alternatively you can provide the private key file yourself
Run:
E.g.:
• This will generate the following key pair in the $HOME/.ssh/ directory:
– id_dsa.myhost67.root (private key / Identity file)
– id_dsa.myhost67.root.pub (Public key)
Confidential – Oracle Internal/Restricted/Highly Restricted 12
ssh-keygen -f $HOME/.ssh/id_dsa.host.user -N ''
ssh-keygen -f $HOME/.ssh/id_dsa.myhost67.user -N ''

Option 1 With internet connection
o When older than 120 days it will prompt you to let it
automatically download newer version from My
Oracle Support
o Can also be specifically triggered with:
Option 2 No internet connection
a) Download the latest version to a shared network
staging location
b) Set environment variable RAT_UPGRADE_LOC:
c) Next time started it will prompt to allow it to
upgrade itself
13
Maintaining
-download
export RAT_UPGRADE_LOC=<staging dir>

Maintaining ORAchk
• Option 3 Applied with Release Update
o Each database release update contains the latest ORAchk version available at time of
creation
o When a database release update is applied the ORAchk zip version contained will be
staged in $ORACLE_HOME/suptools
o The next run of ORAchk it will prompt you to upgrade if the version copied by the
release update is newer than installed – Just like option 2
14

Recommended Usage – Automated Risk Notification
1) Schedule to run in daemon mode weekly or daily and
email report
2) Identify actions easily by viewing automated
comparison of previous runs
3) Act on recommendations
15
Note: Scheduling is only supported on Linux & Solaris

Schedule
?
Hour (0 – 23)
? ? ?
Day of month (1 – 31)
Month (1 – 12)
Day of week (0 – 6)
(0 to 6 are Sunday to Saturday)
example: -set 'AUTORUN_SCHEDULE=8,20 * * 2,5' will schedule runs on Tuesday and Friday at 08:00 & 20:00
AUTORUN_SCHEDULE =
1616
-set “AUTORUN_SCHEDULE=0 3 * * * ;NOTIFICATION_EMAIL=SOME.BODY@COMPANY.COM
-d start
?
Minute (0 – 59)
Optional - default 0 if unset)

• First email will contain the HTML report
Email Notification
17

View Report
• Health score
• Summary of run
• Table of content
• Controls for report features
• Findings
• Recommendations
18

View Report
• Table of content
• Controls for report features
• Checks which passed are not shown by default
• Quickly show or hide checks based on their
status
• Show or hide major sections of the report
• Collapse or expand check findings
• Show check Ids
• Remove the findings from the report – This
doesn’t change the report but optionally hides
findings on a check by check basis
• Display a printable view
19

View Findings
• Check status
• Type of Check
• Check Message
• Where the check was run
• Link to expand details
20

Only the most critical checks shown by default
Only critical checks
shown by default 1
All other checks are
run and available 2
./orachk –profile oratier1
./exachk –profile exatier1
To only run critical checks use the
appropriate profile:

• What to do to solve the problem
• Links to relevant Knowledge docs
• Where recommendation applies
• Where problem doesn’t apply
• Example of data the
recommendation is based on
View Recommendations
22

Review MAA Score Card
• Critical Issues in MAA Scorecard
o All issues reported in “SOFTWARE
MAINTENANCE BEST PRACTICES”
• Software version mapping table
• Installed software versions checked
for noncurrent or incompatible
feature usage
23

Subsequent emails compare results to
previous run
• Easily see if something has changed
• Email attachment has:
o Latest report
o Previous report
o Diff Report
Email Notification
24

Diff overview
• Summary of this run vs previous
Diff Output
25

Differences between each run
Diff Output
26

• Upgrade or Patching
o Detects all databases registered in the
clusterware automatically
o Presents list of databases to check
o Pre Upgrade
 Run during upgrade planning phase
 Asks which version you plan to upgrade to
o Post Upgrade
 Run after upgrade
• Other typical times to run ORAchk
o Machine moves
o Hardware failure / repair
o Problem troubleshooting
o In addition to go-live testing
27
Other Recommended Usage
-u –o pre
-u –o post

Agenda
28
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Set Daemon Options, When, What & Who to Tell
29
–set “<option_1>=<option_1_value>;<option_2>=<option_2_value>;<option_n>=<option_n_value>”
AUTORUN_SCHEDULE
• Schedule when orachk will be run
• Min, Hour, day of month, month of year & day of week
• Comma separate multiple values for same timeframe
• * Wildcard
?
Hour (0 – 23)
? ? ?
Day of month (1 – 31)
Month (1 – 12)
Day of week (0 – 6)
(0 to 6 are Sunday to Saturday)
–set “AUTORUN_SCHEDULE=2 * * 1,3,5”
AUTORUN_FLAGS
• Command line options to be passed through to orachk run
–set “AUTORUN_FLAGS=-profile dba –tag dba”
NOTIFICATION_EMAIL
• Comma separated list of emails to send daemon notifications to
–set “NOTIFICATION_EMAIL=some.person@acompany.com,another.person@acompany.com”
?
Minute (0 – 59)
Optional - default 0 if unset)

Set Daemon Options, Maintenance
PASSWORD_CHECK_INTERVAL
• Frequency in hours of password validation
• When found invalid daemon stops & notifies via log & email
30
–set “<option_1>=<option_1_value>;<option_2>=<option_2_value>;<option_n>=<option_n_value>”
COLLECTION_RETENTION
• Number of days to keep files created by scheduled run, files older than this will be deleted
–set “COLLECTION_RETENTION=30”
–set “PASSWORD_CHECK_INTERVAL=48”

Set Daemon Options, Multiple IDs
• Only one Daemon process should be used across a database cluster
o Multiple daemon profiles can be configured using IDs
o Allows for multiple different types of orachk runs
31
–id <ID> –set “<option>=<value>”

Get Daemon Option
• Find a specific daemon option which has been set
• Use with or without ID
32
–id <ID> –get <option>
Get value of particular option for specific id
–get <option>
Get value of particular option for all IDs

Get All Daemon Options
• Find any Daemon options which have been set
• Used with or without ID
33
–id <ID> –get all
Get value of all options for specific id
–get all
Get value of all options for all IDs

Start Daemon
1. Start the daemon
2. Follow the prompts
34
–d start

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 35
Daemon Information
–d status
Check if the daemon is running:
–d info
Get more detailed information about the daemon

Daemon Next Autorun
–d nextautorun –id <id> –d nextautorun
Find when the next auto run will happen: Find when the next auto run will happen for a specific ID

Daemon Stopping
–d stop_client –d stop
Stop an orachk run mid-flow: Stop the daemon

Auto Restart Daemon
• Restart daemon if
machine restarts
o Uses passwordless ssh
user equivalence to root
38
–initsetup

Restart status of daemon can be queried: If you want to remove restart settings:
39
Auto Restart Daemon, Check Status & Remove Auto Restart
–initcheck –initrmsetup

With or Without the Daemon
• When daemon is running, if orachk is run in on-demand mode it will
connect to the daemon by default and make use of pre-provided prompt
values
o To avoid connecting to the daemon use –nodaemon:
40
–nodaemon

Email Notifications
• Run in ad hoc mode and receive html report via email
• Verify email configuration function
• Use –testemail all to use email addresses stored in daemon configuration
• If the Exadata storage cells are unable to send email use –usecompute, which will
send the email from the database server instead:
or
-sendemail "NOTIFICATION_EMAIL=abc@company.com,xyz@company.com"
-testemail "NOTIFICATION_EMAIL=abc@company.com,xyz@company.com"
-testemail all
41
-testemail "NOTIFICATION_EMAIL=some.person@acompany.com" -usecompute
-sendemail "NOTIFICATION_EMAIL=some.person@acompany.com" -usecompute

Agenda
42
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Health Check Catalog
• Bundled within .zip download
• ORAchk_Health_Check_Catalog.html
• EXAchk_Health_Check_Catalog.html
• Contains all published checks
• Filterable & searchable
• Product Area / Engineered System
• Profiles
• Alert Level
• Release Check Authored
• Platforms
• Privileged User
• Look up check id without running report

Database Checks
• Checks run against all database nodes in the cluster by default
o To specify only a subset of nodes use:
o Only local node:
• Automatically discovers all databases and prompts for which should be checked
o Do not prompt but run all checks on all discovered database:
o Do not prompt and skip all database related checks:
o Only run checks against a subset of databases:
44
–clusternodes <node_1>,<node_2>
–localonly
–dball
–dbnone
–dbnames <db_1>,<db_2>

Grid Infrastructure Only Checks
• Where you only have GI installed or do not want to check the database
45
–nordbms

Output
• EXAchk & ORAchk will output the collection results to the directory it is run from
– Unless run from $ORACLE_HOME/suptools/orachk then output goes to $ORACLE_BASE/orachk
• Output can be directed to a different directory with –output
• Output will be directory and a zip of the same name
46
–output <OUTPUT_DIR>
Output Descriptions
log : various log files
outfiles : collection results checks are based on
reports : subreports used to build the main report
scripts : scripts used during collection
upload : files for upload of collection into database or integration into other tools
orachk_*.html
exachk_*.html
Main HTML report output

Temporary Working Directory
• Temporary files will be created during execution
• Default location is $HOME
• Location can be changed by setting RAT_TMPDIR
• If using sudo access to root from a lower privileged user id, temporary directory must be
reflected in /etc/sudoers file
• Root privilege checks run from root_orachk.sh or root_exachk.sh
– If you want the root script in a different directory to RAT_TMPDIR use: RAT_ROOT_SH_DIR
47
export RAT_TMPDIR=<TEMP_DIR>
<user> ALL=(root) NOPASSWD:<TEMPDIR>/root_[orachk|exachk].sh
oracle ALL=(root) NOPASSWD:/mylocation/root_exachk.sh
oracle ALL=(root) NOPASSWD:/tmp/root_orachk.sh
export RAT_ROOT_SH_DIR=/mylocation

Parallel Execution
• Database collections are executed in parallel
• The default number of slave processes used is calculated automatically
• Default can be changed with –dbparallel <# slave processes> or -dbparallelmax
• Parallel execution can be disabled altogether if required with -dbserial
48
–dbparallel <# slave processes> –dbparallelmax
–dbserial

Tagging, Merging & Comparing Reports
• Collections are typically of the format:
[orachk|exachk]_<dbserver>_<database>_<date>_<timestamp>.html
• Tag collections so output contains another word to help differentiate it:
[orachk|exachk]_<dbserver>_<database>_<date>_<timestamp>_<tag_name>.html
• Merge multiple reports into one with –merge and list of collection directories or zip
files:
• Compare collections with –diff:
49
–merge <collection_1>,<collection_2>
–diff <collection_1>,<collection_2>
–tag <tag_name>

• Profiles provide logical grouping of checks which are about similar topics
• Run only checks in a specific profile
• Run everything except checks in a specific profile
Profiles
50
–profile <profile>
–excludeprofile <profile>

Run or exclude individual checks
• Granular control to execute or exclude a single check
• Ideal for testing new checks or troubleshooting
• Run only specific check(s):
• Exclude a specific check:
• Find check id either from report or Health Check Catalog
51
-check <check_id_1>,<check_id_2>
–excludecheck <check_id_1>,<check_id_2>

Only Run Checks that Previously Failed
1. Generate a health check report
2. Fix the issues identified
3. Generate another health check report verifying only the issues that failed
before
52
-failedchecks <previous_result>

$ ./orachk -fileattr start
CRS stack is running and CRS_HOME is not set. Do you want to set CRS_HOME to
/u01/app/11.2.0.4/grid?[y/n][y]
Checking ssh user equivalency settings on all nodes in cluster
Node mysrv22 is configured for ssh user equivalency for oradb user
Node mysrv23 is configured for ssh user equivalency for oradb user
List of directories(recursive) for checking file attributes:
/u01/app/oradb/product/11.2.0/dbhome_11203
/u01/app/oradb/product/11.2.0/dbhome_11204
orachk has taken snapshot of file attributes for above directories at:
/orahome/oradb/orachk/orachk_mysrv21_20170504_041214
• Track changes to the attributes of important files with –fileattr
– Looks at all files & directories within Grid Infrastructure and Database homes by default
– The list of monitored directories and their contents can be configured to your specific requirements
– Use –fileattr start to start the first snapshot
53
Keep Track of Changes to the Attributes of Important Files
./orachk –fileattr start

• Include other directories with –includedir <directories> using a comma separated list of directories
./orachk –fileattr start includedir “/home/oradb,/etc/oatab”
• Exclude the default discovered directories with –excludediscovery
./orachk –fileattr start includedir “/home/oradb,/etc/oatab” -excludediscovery

Note:
• Use the same arguments with check that you used with start
• Will proceed to perform standard health checks after attribute checking
• File Attribute Changes will also show in HTML report output
$ ./orachk -fileattr check -includedir "/root/myapp/config" -excludediscovery
CRS stack is running and CRS_HOME is not set. Do you want to set CRS_HOME to
/u01/app/12.2.0/grid?[y/n][y]
Checking for prompts on myserver18 for oragrid user...
Checking ssh user equivalency settings on all nodes in cluster
Node myserver17 is configured for ssh user equivalency for root user
List of directories(recursive) for checking file attributes:
/root/myapp/config
Checking file attribute changes...
.
"/root/myapp/config/myappconfig.xml" is different:
Baseline : 0644 oracle root /root/myapp/config/myappconfig.xml
Current : 0644 root root /root/myapp/config/myappconfig.xml
…etc
…etc
• Compare current attributes against first snapshot using –fileattr check
55
./orachk –fileattr check
• Results of snapshot comparison will also
be shown in the HTML report output

• To prevent standard health checking after attribute checking add –fileattronly:
• To use a different snapshot baseline use –baseline:
• To remove all snapshot use –fileattr remove
56
–fileattr check –fileattronly
-fileattr check -baseline <snapshot>
-fileattr remove

Encrypted resulting zip file
• ORAchk and EXAchk can encrypt the resulting collection
zip file
• To use encryption add the option -encryptzip: e.g.
– This will prompt for the password
– Once the zip file is encrypted, the original zip and directory will
be deleted
• To decrypt a zip use:
–profile dba -encryptzip
–decryptzip <zip_filename>
The encrypt/decrypt feature is only supported on Linux and Solaris platforms.

REST Interface
• ORAchk and EXAchk include full REST support, allowing invocation & query over HTTPS
• Oracle REST Data Services (ORDS) is included within the install
• To enable REST:
1. Start ORDS:
2. Start the daemon, using the -ords option:
• Start a full health check run by accessing the URL: https://<host>:7080/ords/tfaml/orachk/start_client
• Run specific profiles: https://<host>:7080/ords/tfaml/orachk/profile/<profile1>,<profile2>
• Run specific checks: https://<host>:7080/ords/tfaml/orachk/check/<check_id>,<check_id>
• Any request will return a job id, which can then be used to query:
– Status: https://<host>:7080/ords/tfaml/orachk/status/<job_id>
– Download result: https://<host>:7080/ords/tfaml/orachk/download/<job_id>
–ordssetup
-d start -ords
The standalone ORDS setup feature utilizes file based user authentication and is provided solely for use in test and development environments.
For production use, the included orachk.jar and ords.war should be deployed and configured.

Agenda
59
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Oracle Health Check Collection Manager Dashboard
60

Dashboard Filters
61
Filter by
Interval
Filter by
configurable
business units Filter by
systems
Click on color coded
area to drill down
Filter by
health score

Most Failures & Warnings
62
Click to see the
recommendation details

Most Failures
63
Click to drill into
failures

Most Warnings
64
Click to drill into
warnings

No difference OR No regression
failed in current collection
At least one regression from
Non-WARNING to WARNING OR
Found WARNING regression in
current collection
At least one regression from
Non-FAIL to FAIL OR Found FAIL
regression in current collection
Non clickable green flag -
Preceding collection not found
Recent Collections
65
Health
Score Warning count
Fail count Info count Pass count
Ignore count

View Collection
66
Collection
Link

View Collection
67
Recommendation

User Defined Checks
• Use as a Health Checking Platform
• You write your own business
specific User Defined Checks
• Collection Manager authoring UI
very similar to Oracle’s internal
authoring tool
• OS or SQL logic
• Generates user_defined_checks.xml
sample in install directory
• Utilizes framework features such as
result output, email notification,
CM storage etc

User Defined Checks
• Have their own profile:
user_defined_checks
• Can be excluded:
-excludeprofile user_defined_checks
• Have their own section of the report
-profile user_defined_checks
• Can be run on their own:

1. First time installation done via the APEX
workspace (either APEX 4.2 or 5.x)
2. Use the sql script applicable for your
APEX version:
– APEX 4.2: CollectionManager_App.sql
– APEX 5.x: Apex5_CollectionManager_App.sql
3. Follow Health Check Collection Manager
installation in the User Guide
4. Login to Collection Manager Application
via a URL like the following:
http://hostname:port/apex/f?p=ApplicationID
http://hostname:port/pls/apex/f?p=ApplicationID
• Collection Manager upgrade done from
orachk / exachk:
• Will determine the APEX version you
have and install the latest applicable
Collection Manager app
• If the Collection Manager schema
changes in the future then ORAchk will
prompt for auto upgrade
70
Setup
-cmupgrade
Note: APEX 4.2 version of CM app exists for backwards compatibility.
New features will only go into the APEX 5 Collection Manager app

Collection Storage Table
71
• Collection zip files are stored in the RCA13_DOCS
table - already created during collection manager
installation
• Provide ORAchk details of where to upload
collection results with –setdbupload all and
complete prompts:
• Get current values with -getdbupload:
• Unset values with –unsetdbupload <parameter>:
-setdbupload all
–unsetdbupload RAT_UPLOAD_PASSWORD
-getdbupload

Store DB Upload Variables in Wallet
• Set all with:
• Set specific variables by specifying comma separated list:
• Unset all with
• Check if variables are set correctly:
-setdbupload all
-setdbupload RAT_UPLOAD_CONNECT_STRING,RAT_UPLOAD_PASSWORD
-unsetdbupload all
-checkdbupload
Other Upload Parameters Not Set
by default
Description
RAT_UPLOAD_USER The user to connect as (default is ORACHKCM)
RAT_UPLOAD_TABLE The table name to store non-zipped collection
results
RAT_PATCH_UPLOAD_TABLE The table name to store non-zipped patch results
RAT_UPLOAD_ORACLE_HOME The ORACLE_HOME used during establishing
connection and uploading.
(Uses GI HOME discovered by ORAchk by default)
RAT_UPLOAD_TABLE &
RAT_PATCH_UPLOAD_TABLE
Only needed if you are using your own custom
application to view collection results, rather than
Collection Manager.

Agenda
73
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Enterprise Manager Integration
•Check results integrated into EM
compliance framework via plugin
•View results in native EM
compliance dashboards
•Related checks grouped into
compliance standards
74
•View targets checked, violations &
average score
•Drill down into compliance standard
to see individual check results
•View break down by target

Setting Up Enterprise Manager Plugin
• The plugin is already installed by default with Enterprise Manager 13.1+
1. Deploy the plugin using the Enterprise Manager Plugin Deployment feature
2. Provision the plugin to setup the daemon
75

• Use Enterprise Manager provisioning
feature and select ORAchk/EXAchk
• After selected this will launch the
provisioning wizard, choose the system
type
76
Provision

Provision
• Provide new or select existing
credentials
• Specify install location
• Select when daemon should be run
77

Provision
• Verify CRS Home
• Finally choose Cell & InfiniBand
configuration
78

Drill into applicable standard and view
individual checks & target status
79
View Results by Compliance Standard
Filter by Exachk%”
Click individual checks for
recommendation details

Agenda
80
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

• The JSON provides many tags to
allow dashboard filtering based on
facts such as:
• Engineered System type
• Engineered System version
• Hardware type
• Node name
• OS version
• Rack identifier
• Rack type
• Database version
• And more...
• Kibana can be used to view health
check compliance across your data
center
• Results can also be filtered based
on any combination of exposed
system attributes
81
JSON Output to Integrate with Kibana, Elastic Search etc

• Results are also output in JSON
format in the upload directory of
the collection
• Writing JSON Results With syslog
1. JSON output results can be sent to the
syslogd Daemon with –syslog option e.g.:
2. Message levels used of “crit”, “err”, “warn”
and “info”
3. You can verify syslog configuration by
running the following commands:
4. Then verify in your configured message
location (e.g. /var/adm/messages) that each
test message was written
82
JSON Result Output
–set “AUTORUN_FLAGS=-syslog”

Integrating With Your Own Application
83

If you don’t use Collection Manager and have your own application which consumes the results
1. Create the tables: auditcheck_result, auditcheck_patch_result & RCA13_DOCS
2. Set default parameters:
– This will prompt you for and set the RAT_UPLOAD_CONNECT_STRING & RAT_UPLOAD_PASSWORD
3. Set optional parameters for RAT_UPLOAD_TABLE & RAT_PATCH_UPLOAD_TABLE
84
Configure Details for Upload of Collection Results
–setdbupload all
-setdbupload RAT_UPLOAD_TABLE,RAT_PATCH_UPLOAD_TABLE
Other Upload Parameters Not Set by default Description
RAT_UPLOAD_USER The user to connect as (default is ORACHKCM)
RAT_UPLOAD_TABLE The table name to store non-zipped collection results
RAT_PATCH_UPLOAD_TABLE The table name to store non-zipped patch results
RAT_UPLOAD_ORACLE_HOME The ORACLE_HOME used during establishing connection and
uploading.
(Uses GI HOME discovered by ORAchk by default)
RAT_UPLOAD_TABLE &
RAT_PATCH_UPLOAD_TABLE
Only needed if you are using your own custom application to view
collection results, rather than Collection Manager.

Agenda
85
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

• View Component Elapsed Times
o Identify if one particular component is slow
• View top 10 Time Consuming Checks
o If some in particular are slow, show check_id
and run again with:
86
Troubleshooting Performance
–excludecheck <check_id>
• Try increasing the number of parallel
slave processes (Note: will increase resource usage)
–dbparallelmax

• If checks are being killed this will be due
to timeout:
• Timeouts can be caused by slow checks or
interactive prompts
• Try increasing default times with applicable
environment variables:
• EBS checks query application data which
can vary hugely in size depending on the
size of your data
oIf you suspect EBS checks try excluding
EBS profile:
oUse RAT_TIMEOUT to increase to a
value where they are no longer killed
87
Troubleshooting Timeouts
–excludeprofile ebs
Environment Variable Timeout Controlled
Default Value
(seconds)
RAT_TIMEOUT
non-root individual
commands
90
RAT_ROOT_TIMEOUT
root userid command
sets
300
RAT_PASSWORDCHECK_TIMEOUT
ssh login DNS
handshake
1

Collecting Debug
• Debugging on-demand runs
• Debugging via Daemon
88
–d start -debug
–debug
• Debug output:

Agenda
89
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Auto Start
90

Automatic Start From Install
• ORAchk will be automatically setup as part of TFA install
– Only on Linux or Solaris
– Only for root install
– Only on non-engineered systems
• Autostart will configure the daemon to restart at 1am every morning to rediscover any environment
changes
• Full Local client run will be triggered at 2am every morning
• Most impactful checks will be run every 6 hours via the oratier1 profile
• Any collections older than 2 weeks will be automatically purged
• To configure Autostart from orachk standalone install use:
– Same constraints apply as with TFA install (only Linux/ Solaris on non-engineered system & run by root)
• Once enabled daemon settings can be changed as per normal
• Remove with:
./orachk -autostart
./orachk -autostop

92

• ORAchk helps ensure that your application is covered by Oracle Application
Continuity
– Identifies any references to deprecated JDNVC concrete classes that need to be
changed
– Analyzes the database operations in the application
– Reports the level of protection, and where applications are not protected
– In places where applications are not protected, ORAchk reports why
93
Concrete Classes
Command-Line Argument Shell Environment
Variable
Usage
–asmhome jarfilename RAT_AC_ASMJAR Point to a version of asm-all-5.0.3.jar that you download from http://asm.ow2.org/
-javahome JDK8dirname RAT_JAVA_HOME Point to the JAVA_HOME directory for a JDK8 installation
-appjar dirname RAT_AC_JARDIR Point to the parent directory name for the code. Will analyze .class files, and recursively .jar files
and directories.

94
Concrete Classes

1. Turn on tracing at either a session level or database level
– To enable for a single application function, run as follows (put this in the callback or before beginRequest so replay is not disabled by setting
events):
– To enable for all session run:
2. Run through the application functions. To report on an application function, it needs to have executed. The more application
functions run, the better the information that the coverage analysis provides.
3. Use ORAchk to analyze the collected database traces and report the level of protection, and where not protected, report why
95
Measure Coverage
Command-Line Argument Shell Environment
Variable
Usage
–asmhome jarfilename RAT_AC_ASMJAR Point to a version of asm-all-5.0.3.jar that you download from http://asm.ow2.org/
-javahome JDK8dirname RAT_JAVA_HOME Point to the JAVA_HOME directory for a JDK8 installation
-apptrc dirname RAT_AC_TRCDIR Directory name containing one or more database server trace files. The trace directory is
generally: $ORACLE_BASE/diag/rdbms/{DB_UNIQUE_NAME}/$ORACLE_SID/trace
alter session set events 'trace [progint_appcont_rdbms]';
alter system set event='trace[progint_appcont_rdbms]' scope = spfile;

• Produces a directory named:
orachk_uname_date_time
• Reports coverage and lists
trace files that have
WARNINGS or FAIL status
96
./orachk -asmhome /tmp/asm-all-5.0.3.jar
-javahome /tmp/jdk1.8.0_40
-apptrc $ORACLE_BASE/diag/rdbms/$ORACLE_SID/trace 3
Measure Coverage
Example output found in orachk_.....html#acchk_scorecard

ORAchk IAM
97

Supported Platforms & Databases
Operating System Database
Linux (Oracle Enterprise Linux/RedHat 5, 6, 7
and SuSE 9,10, 11, 12)
10g R1
Linux on System Z (RedHat 6, 7 and SuSE 12) 11g R1
11g R2
12c
12c R2
Supported Components & Topologies
Components
• Oracle Identity Manager (11.1.2.2.x and 11.1.2.3.x)
• Oracle Access Manager (11.1.2.2.x and 11.1.2.3.x)
• Oracle Unified Directory (11.1.2.2.x and 11.1.2.3.x)
Topologies
• Oracle Identity Manager in single node as well as in multi-node
setups
• Oracle Access Manager + (Any directory)* in single node as
well as in multi-node setups
• Oracle Identity Manager + Oracle Access Manager + (Any
directory)* in single node as well as in multi-node setups
98
ORAchk IAM
• Health checks run on Oracle Unified Directory (OUD) only.
• If other directories are there as well, then ORAchk will skip those directories and
perform health checks on Oracle Access Manager.
• Oracle Access Manager configured in embedded LDAP mode is not supported

Installation
1. Download latest ORAchk IAM version
from 1268927.2
2. Copy the orachk_idm.zip file to the
WebLogic Admin system and extract
3. Ensure JDK 6 or later is set in the
system path or set RAT_JAVA_HOME
Usage
1. Set the environment variable to run the health checks
based on a specific deployment size
2. Run :
3. If database is running on a remote server, then extract
orachk_IAM.zip there
4. Run :
99
ORAchk IAM
• If the oraInst.loc file is not in the default directory then specify the
oraInventory directory using the RAC_INV_LOCAL environment variable.
For example:
export RAT_INV_LOC=/scratch/shared/oracle/oraInventory
Deployment Size Directory User Size
small Close to 100 K
medium Close to 1 million
large Close to 15 million
extralarge Close to 250 million
export RAT_IDM_DEPLOYMENT_SIZE=small
./orachk
./orachk -idmdbruntime

Option Description
-idm –h Displays IDM help
-idmpreinstall Runs all pre-install checks on Identity
Management System
-idmpostinstall Runs all post-install checks on Identity
Management System
-idmruntime Runs all runtime checks on Identity
Management System
-
idmdbpreinstall
Runs pre-install database checks on Identity
Management System
ORAchk IAM - Options
Option Description
-idmpdbostinstall Runs post-install database checks on
Identity Management System
-idmdbruntime Runs runtime database checks on Identity
Management System
-idm_config Passes OAM, OIM, and one of the OUD host
from clusters
-Idmdiscargs Passes arguments to Identity Management
Discovery Tool
-idmhcargs Passes arguments to Identity Management
Healthcheck Tool

ZFS Storage Appliance
101

ZFS Storage Appliance
• To run on one ZFS appliance use the -zfssa option:
• For multiple ZFS appliances specify a comma-delimited list:
102
./orachk -zfssa
./orachk -zfssa node1,node2

Agenda
103
Overview
Autonomous Usage
Troubleshooting
1
2
3
4
5
6
7
8
9

Checks for Cells & Switches
• Run on a subset of cells using –cells
• Run on a subset of switches using -ibswitches
./exachk –cells <cell_1>,<cell_2>
./exachk –ibswitches <switch_1>,<switch_2>
104

Virtualization
1. Run from dom0 on first compute to cover all dom0s, cells and switches
o Install EXAchk into the management domain (DOM0) & run as root
o Will discover all compute nodes, storage servers and InfiniBand switches in the entire InfiniBand fabric
and run EXAchk on all
o If a single machine has been separated into multiple management domains
• Use -clusternodes, -cells, -ibswitches to detail the components of the separate management domains
2. Run once on each virtual cluster
105
./exachk –clusternodes <node_1>, <node_2> –cells <cell_1>,<cell_2> –ibswitches <switch_1>,<switch_2>

• Profiles provide logical grouping of
checks which are about similar topics
• Run only checks in a specific profile
• Run everything except checks in a specific
profile
Profiles
./exachk –profile <profile>
./exachk –excludeprofile <profile>
Profile Description
asm ASM Checks
avdf Audit Vault Configuration checks
clusterware Oracle clusterware checks
control_VM Checks only for Control VM(ec1-vm, ovmm, db, pc1, pc2).
No cross node checks
corroborate Exadata checks needs further review by user to determine
pass or fail
dba DBA Checks
ebs Oracle E-Business Suite checks
eci_healthchecks Enterprise Cloud Infrastructure Healthchecks
ecs_healthchecks Enterprise Cloud System Healthchecks
exatier1 Top tier of Exadata problems with the most severe impact
goldengate Oracle GoldenGate checks
hardware Hardware specific checks for Oracle Engineered systems
maa Maximum Availability Architecture Checks
ovn Oracle Virtual Networking
platinum Platinum certification checks
preinstall Pre-installation checks
prepatch Checks to execute before patching
security Security checks
solaris_cluster Solaris Cluster Checks
storage Oracle Storage Server Checks
switch Infiniband switch checks
sysadmin Sysadmin checks
user_defined_checks Run user defined checks from user_defined_checks.xml
106

Email Notifications
• If the Exadata storage cells are unable to send email use –usecompute, which will
send the email from the database server instead:
or
107
-testemail "NOTIFICATION_EMAIL=some.person@acompany.com" -usecompute
-sendemail "NOTIFICATION_EMAIL=some.person@acompany.com" -usecompute

Checks for Cells & Switches
• Run on a subset of cells using –cells
• Run on a subset of switches using –ibswitches
108
–cells <cell_1>,<cell_2>
–ibswitches <switch_1>,<switch_2>

Virtualization
1. Run from dom0 on first compute to cover all dom0s, cells and switches
o Install EXAchk into the management domain (DOM0) & run as root
o Will discover all compute nodes, storage servers and InfiniBand switches in the entire InfiniBand fabric
and run EXAchk on all
o If a single machine has been separated into multiple management domains
• Use -clusternodes, -cells, -ibswitches to detail the components of the separate management domains
2. Run once on each virtual cluster
109
–clusternodes <node_1>, <node_2> –cells <cell_1>,<cell_2> –ibswitches <switch_1>,<switch_2>

• Exalogic:
– https://docs.oracle.com/cd/E68491_01/OEXUG
/oracle-exalogic.htm#OEXUG-GUID-61B15081-
6424-4F1E-A84C-0E0D8EEF0FF0
• SuperCluster / MiniCluster:
/oracle-supercluster.htm#OEXUG-GUID-
0789AB58-9C67-4F22-98A3-874FAF2EA3E7
110
• Exalytics:
/oracle-exalytics.htm#OEXUG-GUID-D2F1F610-
48EA-4EA6-BF6F-B6A6E252A0E0
• Big Data:
/oracle-big-data.htm#OEXUG-GUID-7534A5C5-
E51E-4341-9D7D-3E2A3215415E
Other Engineered System Specifics

Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
111

Oracle ORAchk & EXAchk overview

Oracle ORAchk & EXAchk overview

Recommended

More Related Content

What's hot (18)

Similar to Oracle ORAchk & EXAchk overview (20)

Recently uploaded (20)

Oracle ORAchk & EXAchk overview