1. Create nodelist.pt file path /opt/softwareag/API_Portal/server.

Update the fi
le with content as below .

add node n1 dkaint-pc0c91.corp.lego.com @18003 Clous g3h31m

add node n2 dkaint-pc0c92.corp.lego.com @18003 Clous g3h31m
add node n3 dkaint-pc0c93.corp.lego.com @18003 Clous g3h31m
set current node n1
2. Ensure that Cloud agent is running by Checking status as below .
[saguser@dkaint-pc0c91 bin]$ ./CloudAgentApp.sh status
Softare AG API-Portal 9.9 is running: PID:1764, Wrapper:STARTED, Java:STARTED
3. Go to path /opt/softwareag/API_Portal/server/acc & Run command as $./acc.sh
-n /opt/softwareag/API_Portal/server/nodelist.pt -c /opt/softwareag/API_Portal/
server/generated.apptypes.cfg
[saguser@dkaint-pc0c91 acc]$ ./acc.sh -n /opt/softwareag/API_Portal/server/nodel
ist.pt -c /opt/softwareag/API_Portal/server/generated.apptypes.cfg
Executing command "add node n1 dkaint-pc0c91.corp.lego.com @18003 Clous g3h31m"
Agent at dkaint-pc0c91.corp.lego.com:18003 added successfully with logical node
name n1.
Executing command "add node n2 dkaint-pc0c92.corp.lego.com @18003 Clous g3h31m"
Agent at dkaint-pc0c92.corp.lego.com:18003 added successfully with logical node
name n2.
Executing command "add node n3 dkaint-pc0c93.corp.lego.com @18003 Clous g3h31m"
Agent at dkaint-pc0c93.corp.lego.com:18003 added successfully with logical node
name n3.
Executing command "set current node n1"
Node n1 is set as current node.
Validation of the nodes n1, n2 and n3 completed without errors.
There were warnings:
One warning found related to several or all nodes:
There are 3 Zookeeper ensembles across the node(s) n1, n2 and n3.
ACC+ n1>

ACC+ n1>on all nodes list

On node n1 7 runnables are installed.
zoo_m : STOPPED (com.aris.runnables.zookeeper-run-prod-99.0.0)
postgres_m : STOPPED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
loadbalancer_m : STOPPED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)
On node n2 7 runnables are installed.
zoo_m : STOPPED (com.aris.runnables.zookeeper-run-prod-99.0.0)
postgres_m : STOPPED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
loadbalancer_m : STOPPED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)
On node n3 7 runnables are installed.
zoo_m : STOPPED (com.aris.runnables.zookeeper-run-prod-99.0.0)
postgres_m : STOPPED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
loadbalancer_m : STOPPED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)

ACC+ n1>list nodes

3 nodes:
n1 : dkaint-pc0c91.corp.lego.com (18003) OK
n2 : dkaint-pc0c92.corp.lego.com (18003) OK
n3 : dkaint-pc0c93.corp.lego.com (18003) OK
*************************************************************************
4. Cleaning up the unnecessary runnables
The unneeded runnables are removed by deconfiguring them. Do the following in AC
C, the last command might take some time since a backup is usually created as sa
fety precaution :
ACC+ n1>on n1 deconfigure zoo_m
ACC+ n1>on n1 deconfigure couchdb_m
ACC+ n1>on n1 deconfigure cloudsearch_m
ACC+ n1>on n1 deconfigure apiportalbundle_m
For the nodes n2 and n3 issue the following commands in ACC:
ACC+ n1>on n2 deconfigure zoo_m
ACC+ n1>on n2 deconfigure postgres_m
ACC+ n1>on n2 deconfigure loadbalancer_m
ACC+ n1>on n3 deconfigure zoo_m
ACC+ n1>on n3 deconfigure postgres_m
ACC+ n1>on n3 deconfigure loadbalancer_m

Output: ACC+ n1>on n1 deconfigure zoo_m

This will completely remove the runnable zoo_m including any persistent data it
might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable zoo_m on node n1.
ACC+ n1>on n1 deconfigure couchdb_m
This will completely remove the runnable couchdb_m including any persistent data
it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable couchdb_m on node n1.
ACC+ n1>on n1 deconfigure cloudsearch_m
This will completely remove the runnable cloudsearch_m including any persistent
data it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable cloudsearch_m on node n1.
ACC+ n1>on n1 deconfigure apiportalbundle_m
This will completely remove the runnable apiportalbundle_m including any persist
ent data it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable apiportalbundle_m on node n1.
ACC+ n1>on n2 deconfigure zoo_m
This will completely remove the runnable zoo_m including any persistent data it
might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable zoo_m on node n2.
ACC+ n1>on n2 deconfigure postgres_m
This will completely remove the runnable postgres_m including any persistent dat
a it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable postgres_m on node n2.
ACC+ n1>on n2 deconfigure loadbalancer_m
This will completely remove the runnable loadbalancer_m including any persistent
data it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable loadbalancer_m on node n2.
ACC+ n1>on n3 deconfigure zoo_m
This will completely remove the runnable zoo_m including any persistent data it
might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable zoo_m on node n3.
ACC+ n1>on n3 deconfigure postgres_m
This will completely remove the runnable postgres_m including any persistent dat
a it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable postgres_m on node n3.
ACC+ n1>on n3 deconfigure loadbalancer_m
This will completely remove the runnable loadbalancer_m including any persistent
data it might hold.
Are you sure you want to continue? (Y/N)
y
Successfully deconfigured the runnable loadbalancer_m on node n3.
ACC+ n1>
################################################################################
#################
5. Zookeeper
Create a zookeeper cluster via
ACC+ n1>on n1 add zk
ACC+ n1>on n2 add zk
ACC+ n1>on n3 add zk
ACC+ n1>commit zk changes
You should see the following configuration via the ACC command after you started
the zoo runnables:
ACC+ n1>list zk instances
3 Zookeeper instances:
Node InstID MyID State Cl Port Port A Port B Type
n1 zoo0 1 STARTED 2181 2888 3888 Master
n2 zoo0 2 STARTED 2181 2888 3888 Master
n3 zoo0 3 STARTED 2181 2888 3888 Master

Output:
ACC+ n1>on n1 add zk
Zookeeper master instance will be added on node n1.
ACC+ n1>on n2 add zk
Zookeeper master instance will be added on node n2.
ACC+ n1>on n3 add zk
Zookeeper master instance will be added on node n3.
ACC+ n1>commit zk changes
Zookeeper ensemble changes committed.
ACC+ n1>list zk instances
3 Zookeeper instances:
Node InstID MyID State Cl Port Port A Port B Type
n1 zoo0 1 STARTED 2181 2888 3888 Master
n2 zoo0 2 STARTED 2181 2888 3888 Master
n3 zoo0 3 STARTED 2181 2888 3888 Master
***************************************************************************
ACC+ n1>on all nodes list
On node n1 4 runnables are installed.
postgres_m : STOPPED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
loadbalancer_m : STOPPED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n2 5 runnables are installed.
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n3 5 runnables are installed.
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
********************************************************************************
***

6. Elasticsearch
Reconfigure the three elasticsearch runnables to form a cluster via:
ACC+ n1>on n1 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAS
TICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICS
EARCH.sonian.elasticsearch.zookeeper.client.host
ACC+ n1>on n2 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAST
ICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICSE
ARCH.sonian.elasticsearch.zookeeper.client.host
ACC+ n1>on n3 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAST
ICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICSE
ARCH.sonian.elasticsearch.zookeeper.client.host

Output:
ACC+ n1>on n1 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAS
TICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICS
EARCH.sonian.elasticsearch.zookeeper.client.host
Successfully delta-reconfigured runnable elastic_m on node n1.
Effective parameters after delta-reconfigure:
ELASTICSEARCH.cluster.name=apiportal
ELASTICSEARCH.discovery.type=com.sonian.elasticsearch.zookeeper.discover
y.ZooKeeperDiscoveryModule
ELASTICSEARCH.discovery.zen.minimum_master_nodes=2
ELASTICSEARCH.discovery.zen.ping.multicast.enabled=false
ELASTICSEARCH.http.compression=true
ELASTICSEARCH.http.port=18044
ELASTICSEARCH.index.number_of_replicas=1
ELASTICSEARCH.index.number_of_shards=3
ELASTICSEARCH.node.local=false
ELASTICSEARCH.rest.action.multi.allow_explicit_index=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.discovery.state_publishing.
enabled=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.settings.enabled=false
ELASTICSEARCH.transport.tcp.compress=true
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=9676
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-XX\:CMSInitiatingOccupancyFraction=75
JAVA-XX\:\+HeapDumpOnOutOfMemoryError=/enabled
JAVA-XX\:\+UseCMSInitiatingOccupancyOnly=/enabled
JAVA-XX\:\+UseConcMarkSweepGC=/enabled
JAVA-XX\:\+UseParNewGC=/enabled
JAVA-Xmx=1g
JAVA-Xss=256k
JAVA_OPTS=-server
ACC+ n1>on n2 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAST
ICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICSE
ARCH.sonian.elasticsearch.zookeeper.client.host
Successfully delta-reconfigured runnable elastic_m on node n2.
Effective parameters after delta-reconfigure:
ELASTICSEARCH.cluster.name=apiportal
ELASTICSEARCH.discovery.type=com.sonian.elasticsearch.zookeeper.discover
y.ZooKeeperDiscoveryModule
ELASTICSEARCH.discovery.zen.minimum_master_nodes=2
ELASTICSEARCH.discovery.zen.ping.multicast.enabled=false
ELASTICSEARCH.http.compression=true
ELASTICSEARCH.http.port=18044
ELASTICSEARCH.index.number_of_replicas=1
ELASTICSEARCH.index.number_of_shards=3
ELASTICSEARCH.node.local=false
ELASTICSEARCH.rest.action.multi.allow_explicit_index=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.discovery.state_publishing.
enabled=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.settings.enabled=false
ELASTICSEARCH.transport.tcp.compress=true
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=9676
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-XX\:CMSInitiatingOccupancyFraction=75
JAVA-XX\:\+HeapDumpOnOutOfMemoryError=/enabled
JAVA-XX\:\+UseCMSInitiatingOccupancyOnly=/enabled
 JAVA-XX\:\+UseConcMarkSweepGC=/enabled
JAVA-XX\:\+UseParNewGC=/enabled
JAVA-Xmx=1g
JAVA-Xss=256k
JAVA_OPTS=-server
ACC+ n1>on n3 reconfigure elastic_m +ELASTICSEARCH.cluster.name = apiportal +ELA
STICSEARCH.discovery.zen.minimum_master_nodes=2 -zookeeper.connect.string +ELAST
ICSEARCH.node.local = false +ELASTICSEARCH.index.number_of_replicas=1 -ELASTICSE
ARCH.sonian.elasticsearch.zookeeper.client.host
Successfully delta-reconfigured runnable elastic_m on node n3.
Effective parameters after delta-reconfigure:
ELASTICSEARCH.cluster.name=apiportal
ELASTICSEARCH.discovery.type=com.sonian.elasticsearch.zookeeper.discover
y.ZooKeeperDiscoveryModule
ELASTICSEARCH.discovery.zen.minimum_master_nodes=2
ELASTICSEARCH.discovery.zen.ping.multicast.enabled=false
ELASTICSEARCH.http.compression=true
ELASTICSEARCH.http.port=18044
ELASTICSEARCH.index.number_of_replicas=1
ELASTICSEARCH.index.number_of_shards=3
ELASTICSEARCH.node.local=false
ELASTICSEARCH.rest.action.multi.allow_explicit_index=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.discovery.state_publishing.
enabled=false
ELASTICSEARCH.sonian.elasticsearch.zookeeper.settings.enabled=false
ELASTICSEARCH.transport.tcp.compress=true
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=9676
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-XX\:CMSInitiatingOccupancyFraction=75
JAVA-XX\:\+HeapDumpOnOutOfMemoryError=/enabled
JAVA-XX\:\+UseCMSInitiatingOccupancyOnly=/enabled
JAVA-XX\:\+UseConcMarkSweepGC=/enabled
JAVA-XX\:\+UseParNewGC=/enabled
JAVA-Xmx=1g
JAVA-Xss=256k
JAVA_OPTS=-server

7. Database
we will be using the pre-configured PostgreSQL database, make sure it knows abou
t all zookeeper cluster members and accepts connections from all locations:
on n1 reconfigure postgres_m -zookeeper.connect.string +postgresql.listen_addre
sses = "'*'"
Output: ACC+ n1>on n1 reconfigure postgres_m -zookeeper.connect.string +postgre
sql.listen_addresses = "'*'"
Successfully delta-reconfigured runnable postgres_m on node n1.
Effective parameters after delta-reconfigure:
dbsp.initialSize=0
dbsp.jmxEnabled=true
dbsp.maxIdle=5
dbsp.minIdle=0
dbsp.removeAbandonedTimeout=600
dbsp.rollbackOnReturn=true
plugin.apptypes=db, postgresql-ecp
plugin.max.retries.after.restart=1
 plugin.ping.interval.msec=30000
postgresql.listen_addresses='*'
postgresql.max_connections=200
postgresql.port=18047
8. CloudSearch
Define two cloudsearch instances where each one belongs to a different data cent
er:
on n2 reconfigure cloudsearch_m -zookeeper.connect.string +zookeeper.application
.instance.datacenter = n2
on n3 reconfigure cloudsearch_m -zookeeper.connect.string +zookeeper.application
.instance.datacenter = n3

Output: ACC+ n1>ACC+ n1>on n2 reconfigure cloudsearch_m -zookeeper.connect.strin

g +zookeeper.application.instance.datacenter = n2
Successfully delta-reconfigured runnable cloudsearch_m on node n2.
Effective parameters after delta-reconfigure:
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=18020
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-XX\:\+HeapDumpOnOutOfMemoryError=/enabled
JAVA-Xmx=2g
JAVA_OPTS=-server -Xrunjdwp:transport=dt_socket,address=18021,server=y,s
uspend=n
zookeeper.application.instance.datacenter=n2
zookeeper.application.instance.port=18045
ACC+ n1>ACC+ n1>on n3 reconfigure cloudsearch_m -zookeeper.connect.string +zooke
eper.application.instance.datacenter = n3
Successfully delta-reconfigured runnable cloudsearch_m on node n3.
Effective parameters after delta-reconfigure:
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=18020
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-XX\:\+HeapDumpOnOutOfMemoryError=/enabled
JAVA-Xmx=2g
JAVA_OPTS=-server -Xrunjdwp:transport=dt_socket,address=18021,server=y,s
uspend=n
zookeeper.application.instance.datacenter=n3
zookeeper.application.instance.port=18045
9. CouchDB
Reconfigure the couchdb runnable on the nodes n2 and n3 via
on n2 reconfigure couchdb_m -zookeeper.connect.string
on n3 reconfigure couchdb_m -zookeeper.connect.string
Output:
ACC+ n1>on n2 reconfigure couchdb_m -zookeeper.connect.string
Successfully delta-reconfigured runnable couchdb_m on node n2.
Effective parameters after delta-reconfigure:
COUCHDB.httpd.port=18046
ACC+ n1>on n3 reconfigure couchdb_m -zookeeper.connect.string
Successfully delta-reconfigured runnable couchdb_m on node n3.
Effective parameters after delta-reconfigure:
 COUCHDB.httpd.port=18046

10. API-Portal Bundle

Reconfigure the apiportalbundle runnable on the nodes n2 and n3 via
on n2 reconfigure apiportalbundle_m -zookeeper.connect.string
on n3 reconfigure apiportalbundle_m -zookeeper.connect.string
Output:
ACC+ n1>on n2 reconfigure apiportalbundle_m -zookeeper.connect.string
Successfully delta-reconfigured runnable apiportalbundle_m on node n2.
Effective parameters after delta-reconfigure:
CATALINA_AJP_PORT=18155
CATALINA_AJP_packetSize=32768
CATALINA_HTTP_maxHttpHeaderSize=32768
CATALINA_PORT=18113
JAVA-DapiPortal.license.path="/opt/softwareag/API_Portal/license.xml"
JAVA-DapiPortal.path="/opt/softwareag/API_Portal"
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=18022
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-Xmx=12G
JAVA_OPTS=-server -Xrunjdwp:transport=dt_socket,address=18023,server=y,s
uspend=n
ajpNumWorkerThreads=100
ecp.env.platform.name=WEB_METHODS_API_PORTAL_RL
httpNumWorkerThreads=100
installation.type=API_PORTAL
zookeeper.bundle.type=ecp
zookeeper.connect.retry.time.ms=30000
zookeeper.session.timeout.ms=60000
ACC+ n1>on n3 reconfigure apiportalbundle_m -zookeeper.connect.string
Successfully delta-reconfigured runnable apiportalbundle_m on node n3.
Effective parameters after delta-reconfigure:
CATALINA_AJP_PORT=18155
CATALINA_AJP_packetSize=32768
CATALINA_HTTP_maxHttpHeaderSize=32768
CATALINA_PORT=18113
JAVA-DapiPortal.license.path="/opt/softwareag/API_Portal/license.xml"
JAVA-DapiPortal.path="/opt/softwareag/API_Portal"
JAVA-Dcom.sun.management.jmxremote=/enabled
JAVA-Dcom.sun.management.jmxremote.authenticate=false
JAVA-Dcom.sun.management.jmxremote.port=18022
JAVA-Dcom.sun.management.jmxremote.ssl=false
JAVA-Xmx=12G
JAVA_OPTS=-server -Xrunjdwp:transport=dt_socket,address=18023,server=y,s
uspend=n
ajpNumWorkerThreads=100
ecp.env.platform.name=WEB_METHODS_API_PORTAL_RL
httpNumWorkerThreads=100
installation.type=API_PORTAL
zookeeper.bundle.type=ecp
zookeeper.connect.retry.time.ms=30000
zookeeper.session.timeout.ms=60000

11. Loadbalancer
Reconfigure the loadbalancer on n1 to point to all three zookeeper cluster membe
rs via
on n1 reconfigure loadbalancer_m -zookeeper.connect.string
Output: ACC+ n1>on n1 reconfigure loadbalancer_m -zookeeper.connect.string
Successfully delta-reconfigured runnable loadbalancer_m on node n1.
Effective parameters after delta-reconfigure:
HTTPD.LimitRequestFieldSize=32768
HTTPD.access.root=granted
HTTPD.keepalive=on
HTTPD.modjk.max_packet_size=32768
HTTPD.modjk.stickySessions.abs=true
HTTPD.modjk.stickySessions.ads=true
HTTPD.modjk.stickySessions.ecp=true
HTTPD.modjk.stickySessions.processboard=true
HTTPD.modjk.stickySessions.umc=true
HTTPD.port=18101
HTTPD.servername=dkaint-pc0c91.corp.lego.com
HTTPD.ssl.port=18102
appcontext.abs=abs
appcontext.cop=/
appcontext.ecp=collaboration
plugin.ping.search.for.processes=false
zookeeper.application.instance.host=dkaint-pc0c91.corp.lego.com
12. Change the Startup Order of the Runnables
Issue the following commands:
ACC+ n1>on n1 set runnable.order = "zoo0 < (elastic_m, postgres_m) < loadbalance
r_m"
ACC+ n1>on n2 set runnable.order = "zoo0 < (elastic_m, couchdb_m) < cloudsearch_
m < apiportalbundle_m"
ACC+ n1>on n3 set runnable.order = "zoo0 < (elastic_m, couchdb_m) < cloudsearch_
m < apiportalbundle_m"
output: ACC+ n1>on n1 set runnable.order = "zoo0 < (elastic_m, postgres_m) < loa
dbalancer_m"
Successfully set configuration parameter runnable.order to zoo0 < (elastic_m, po
stgres_m) < loadbalancer_m on node n1
ACC+ n1>on n2 set runnable.order = "zoo0 < (elastic_m, couchdb_m) < cloudsearch_
m < apiportalbundle_m"
Successfully set configuration parameter runnable.order to zoo0 < (elastic_m, co
uchdb_m) < cloudsearch_m < apiportalbundle_m on node n2
ACC+ n1>on n3 set runnable.order = "zoo0 < (elastic_m, couchdb_m) < cloudsearch_
m < apiportalbundle_m"
Successfully set configuration parameter runnable.order to zoo0 < (elastic_m, co
uchdb_m) < cloudsearch_m < apiportalbundle_m on node n3
ACC+ n1>

Status of all runnables on all nodes :

ACC+ n1>on all nodes list

On node n1 4 runnables are installed.
postgres_m : STOPPED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
loadbalancer_m : STOPPED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n2 5 runnables are installed.
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n3 5 runnables are installed.
couchdb_m : STOPPED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STOPPED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STOPPED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STOPPED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)

################################################################################
###############################################

Scripting :

How to start everything up

Use the provided start up script
#
# start Zookeeper Ensemble
#
on n1 start zoo0
on n2 start zoo0
on n3 start zoo0
on n1 wait for STARTED of zoo0
on n2 wait for STARTED of zoo0
on n3 wait for STARTED of zoo0
#
# start ElasticSearch Cluster
#
on n1 start elastic_m
on n2 start elastic_m
on n3 start elastic_m
on n1 wait for STARTED of elastic_m
on n2 wait for STARTED of elastic_m
on n3 wait for STARTED of elastic_m
#
# start CouchDB
#
on n2 start couchdb_m
on n3 start couchdb_m
on n2 wait for STARTED of couchdb_m
on n3 wait for STARTED of couchdb_m
#
# start PostgreSQL Database
#
on n1 start postgres_m
on n1 wait for STARTED of postgres_m
#
# start CloudSearch
#
on n2 start cloudsearch_m
on n3 start cloudsearch_m
on n2 wait for STARTED of cloudsearch_m
on n3 wait for STARTED of cloudsearch_m
#
# start API-Portal Bundle
#
on n2 start apiportalbundle_m
on n3 start apiportalbundle _m
on n2 wait for STARTED of apiportalbundle _m
on n3 wait for STARTED of apiportalbundle _m
#
# finally, start loadbalancer
#
on n1 start loadbalancer_m
on n1 wait for STARTED of loadbalancer_m

################################################################################
############
ACC+ n1>on all nodes list
On node n1 4 runnables are installed.
postgres_m : STARTED (com.aris.runnables.PostgreSQL-run-prod-99.0.0)
elastic_m : STARTING (com.aris.runnables.elasticsearch-run-prod-99.0.0)
loadbalancer_m : STARTED (com.aris.runnables.httpd.httpd-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n2 5 runnables are installed.
couchdb_m : STARTED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STARTED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STARTED (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STARTED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)
On node n3 5 runnables are installed.
couchdb_m : STARTED (com.aris.runnables.couchdb-run-prod-99.0.0)
cloudsearch_m : STARTED (com.aris.cip.y-cloudsearch-run-prod-99.0.0)
elastic_m : STARTING (com.aris.runnables.elasticsearch-run-prod-99.0.0)
apiportalbundle_m : STARTED (com.aris.bundles.apiportalbundle-run-prod-99.0.0)
zoo0 : STARTED (com.aris.runnables.zookeeper-run-prod-99.0.0)

################################################################################
##################

Update , PostgreSQL Database:

1. cd /opt/softwareag/API_Portal/server/bin/work/work_postgres_m/pgworkdata
[saguser@dkaint-qc0c91 pgworkdata]$ more pg_hba.conf

host all all 0.0.0.0/0 md5

2. cd /opt/softwareag/API_Portal/server/bin/work/work_elastic_m/elasticsearch/co
nfig
 change file elasticsearch.yml
discovery.zen.ping.unicast.hosts: ["dkaint-pc0c91.corp.lego.com:20000","dkaint-p
c0c92.corp.lego.com:20000","dkaint-pc0c93.corp.lego.com:20000"]
node.master: true
network.host: 0.0.0.0
network.publish_host: dkaint-pc0c91.corp.lego.com

3. on n1 show instance elastic_m health

4. on n1 show instance zoo0
5. zk ls /es/clusters/apiportal/nodes
**6. on n1 show instance elastic_m verbose
This command gives full length of parameter values.

Health Checkup
http://dkaint-pc0c93.corp.lego.com:18044/_cluster/health?pretty=true

https://itrac.eur.ad.sag/browse/YAP-2805
We went through the provided information. We did not find any hints related to p
ort availability problems.
Instead we have found a difference in the elastic search configuration. In the i
nternal sample configurations the following parameter is set to true:
sonian.elasticsearch.zookeeper.discovery.state_publishing.enabled=true
In the customer configuration this parameter is set to false. Customer should se
t the parameter to true and try again.
http://dkaint-pc0c91.corp.lego.com:18051/_utils/fauxton/