CloudStack for Java User Group
Download as PPT, PDF4 likes2,315 views
Apache CloudStack is an open-source Infrastructure as a Service (IaaS) platform initiated by VMops, later acquired by Citrix, and accepted into the Apache Incubator in 2012. With a focus on community-driven development and contributions from various organizations, CloudStack supports complex networking and is hypervisor-agnostic, allowing for extensive customization and integration. The platform aims to facilitate fast adoption in enterprise environments by providing an open-source solution desired by users and developers alike.
![Making API calls
Can be authenticated or not.
HTTP call.
User Keys can be generated via the GUI
Base url: http://<manager-host>:8080/client/api?….
def make_request(requests, secretKey):
request = zip(requests.keys(), requests.values())
request.sort(key=lambda x: str.lower(x[0]))
requestUrl = "&".join(["=".join([r[0], urllib.quote_plus(str(r[1]))]) for r in request])
hashStr = "&".join(["=".join([str.lower(r[0]),
str.lower(urllib.quote_plus(str(r[1]))).replace("+", "%20")]) for r in request])
sig = urllib.quote_plus(base64.encodestring(hmac.new(secretKey, hashStr,
hashlib.sha1).digest()).strip())
print "Signature: %s"%sig
requestUrl += "&signature=%s"%sig
print requestUrl
if __name__ == '__main__':
requests = {
"apiKey": "BRZ5j4E8O4di2MZWnQsYBLThCrTGO-LGeZaMjsnvelkHuY5P8FdTnluNZTDQhCUy-
wqeJzk8EAc_NbcZxTF_FA",
"response" : "json",
"command" : "listZones"
}
secretKey = "bFlx2llt3OmM4AiHzfwV1ZbuJ5tsv6hAx6IeM32CkM-obCA77BRwBr3_yQ0bO1-kdZyfD3-lY6khsXCx18n3Mw"
make_request(requests, secretKey)](https://image.slidesharecdn.com/cs-genevajug-121030093641-phpapp01/85/CloudStack-for-Java-User-Group-27-320.jpg)
![Testing Framework –for the PyUG
[environment]
• Marvin is a Python dns=10.147.28.6
mshost=10.147.39.69
based framework to run mysql.host=10.147.39.69
tests against a
[cloudstack]
private.gateway=10.147.40.1
CloudStack install private.pod.startip=10.147.41.121
private.pod.endip=10.147.41.160
• Could be used as a private.netmask=255.255.254.0
public.gateway=10.147.40.1
simulator of a public.vlan.startip=10.147.41.162
datacenter
public.vlan.endip=10.147.41.200
public.netmask=255.255.254.0
• Used to configure an
hypervisor=XenServer
host=10.147.40.10
infrastructure on a mgt host.password=password
#storage pools
server primary.pool=nfs://10.147.28.7:/expo
rt/home/automation/sadhu/primary
secondary.pool=nfs://10.147.28.6:/ex
port/home/automation/sadhu/secondary](https://image.slidesharecdn.com/cs-genevajug-121030093641-phpapp01/85/CloudStack-for-Java-User-Group-36-320.jpg)
CloudStack for Java User Group
- 1. Apache CloudStack Sebastien Goasguen GenevaJUG, Oct 30th Geneve, Suisse
- 2. Info • Apache incubator project • http://www.cloudstack.org • http://incubator.apache.org/cloudstack/ • #cloudstack on irc.freenode.net • @cloudstack on Twitter • http://www.slideshare.net/cloudstack • http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !
- 3. A bit of History • Original company VMOPs (2008) – Founded by Sheng Liang former lead dev on JVM • Open source (GPLv3) as CloudStack • Acquired by Citrix (July 2011) • Relicensed under ASL v2 April 3, 2012 • Accepted as Apache Incubating Project April 16, 2012 • First Apache (ACS 4.0) release expected Sept 26th
- 5. Apache Process • 100% community driven • New ideas, decisions only taken on mailing lists. Votes taken by community • Project led by Project Management Committee (PMC): – http://www.apache.org/dev/pmc.html – http://incubator.apache.org/guides/committer.html • Non committers get invited as committers: – http://community.apache.org/newcommitter.html
- 7. First Release almost out • Apache CloudStack 4.0 • Check out the testing procedure: – https://cwiki.apache.org/CLOUDSTACK/cloudstack -40-test-procedure.html • Or watch my amazing screencast: – http://vimeo.com/52150218
- 8. Contributions from outside Citrix Examples: Sungard: Announced that 6 developers were joining the Apache project Schuberg Philis: Big contribution in building/packaging and Nicira support Go Daddy: Early proto of Maven building Caringo: Support for own object store
- 9. CloudStack and Citrix • CloudPlatform powered by CloudStack • Feature parity, moving to CloudStack being upstream • Enterprise support • Enterprise training • Main contributor to Apache CloudStack, up to now
- 10. Why make it Open Source ? • Large partner ecosystem needed a way to integrate quickly • Users drive adoption in early markets – providing shortest path to adoption is open source • Open source communities are driving integrations and standards • Faster time to market, short feedback loop from user community • Enterprise ready and wanted an open source solution for customers to develop on.
- 11. What Does it do ? • Open source Infrastructure as a Service (IaaS) platform. • Data Center orchestrator • Hypervisor agnostic (with addition of bare metal provisioning) • Support complex enterprise networking (e.g Firewall, load balancer, VPN, VPC…) • Multi-tenant
- 12. Other OSS IaaS Players
- 13. A Very Flexible IaaS Platform Compute Hypervisor Storage Block & Object Primary Storage Secondary Storage Network Network & Network Services http://www.slideshare.net/cloudstack/cloudstack-architecture
- 14. Architecture / Language • Java application • Tomcat6, Axis2, Maven build + ant – Ant going away in 4.1 • Moving towards a plugin architecture – Cocoon ? Spring ? • Collaboration Conference, Nov 30th -Dec 2nd – http://collab12.cloudstack.org/
- 15. Build and Run in 4.1 • git clone https://git-wip- us.apache.org/repos/asf/incubat or-cloudstack.git • mvn clean • mvn install • mvn –P developer –pl developer –Ddeploydb • mvn –pl :cloud-client-ui jetty:run
- 16. Cloud Interactions OVM Cluster Primary Storage Ap i Xen vcenter vcenter Monitoring Primary CS API Clu I ste AP vSphere Cluster Storage r M n ter End End gm vCe t User UI User UI Primary XS Cluster Storage Admin Admin Clustered Clustered CloudStack XAPI UI UI CloudStack Domain Domain CS Admin & CloudStack CloudStack CloudStack CloudStack End-user API Primary Admin Admin UI UI Management Management JSON KVM Cluster Storage Server Server NetConf Juniper SRX Cloud user Nitro API {API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console Console {ec2 API client } ProxyConsole VM Proxy VM Proxy VM Proxy VM NFS MySQL MySQL Server Sec. Storage Sec. Storage NFS NFS {Proxied} SSH Sec. VM Storage Sec. Storage VM Ajax Ajax HTTPS VM VM Console Console Router VM Router VM HTTP (Template Download) Router VM Router VM HTTP (Template Copy) Router VM Router VM Cloud user HTTP (Swift) http://www.slideshare.net/cloudstack/cloudstack-architecture
- 17. Terminology Zone: Availability zone, aka Regions. Could be worldwide. Different data centers Pods: Racks or aisles in a data center Clusters: Group of machines with a common type of Hypervisor Host: A Single server Primary Storage: Shared storage across a cluster Secondary Storage: Shared storage in a single Zone
- 18. Storage • Primary Storage: – Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI… – Holds disk images of running VMs – Support for CEPH with KVM hypervisors • Secondary Storage: – Available across the zone – Holds snapshots and templates (image repo) – Can use Openstack swift or any object store (Gluster FS…) – New support for Caringo • Can use NFS for both to start • Storage Abstraction refactoring underway
- 19. Networking • Extremely flexible to: – Provide isolation with VLANs – Provide isolation at L3 with shared L2 (scalability) – Support hardware devices that exposes API – Deployed on existing networking infrastructure – Support new networking paradigm (SDN) • Support for Nicira Virtual P • Extensive use of Open VSwitch
- 20. Physical Network Operations Users Admin and Cloud API CloudStack Mgmt Server Cluster Router MySQL Load Balancer Availability Zone L3 Core Switch Access Layer Switches … Secondary Servers Storage … … … … Pod 1 Pod 2 Pod 3 Pod N Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
- 21. Layer-2 Guest Virtual Network CS Virtual Router provides Network Services External Devices provide Network Services Network Hardware exposing API can be controlled Guest Virtual Network 10.1.1.1/8 Guest Virtual Network 10.1.1.1/8 VLAN 100 VLAN 100 Public Public Network/Inter Network/Inter net Guest net Guest Guest Guest Private IP VM 11 Public IP 10.1.1.1 VM 11 10.1.1.1 VM 10.1.1.111 VM Gateway 65.37.141.11 Juniper Juniper Public IP 1 SRX SRX address 65.37.141.11 CS Firewall Firewall CS 10.1.1.1 Guest Guest Guest Guest Virtual Virtual 10.1.1.3 VM 22 VM 10.1.1.3 VM 22 VM Router Router Public IP Private IP DHCP, DNS 65.37.141. NetScaler 10.1.1.112 NetScaler NAT Guest 112 Load Guest Guest Load Guest Load Balancing 10.1.1.4 VM 33 Blancer VM 33 VM Blancer 10.1.1.4 VM VPN Guest Guest Guest Guest 10.1.1.5 VM 44 VM 10.1.1.5 VM 44 VM CS CS DHCP, Virtual Virtual Router Router DNS Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
- 22. L3 isolation • To go beyond the limitation of VLANs (4096 vlans per switch) • Move to L3 isolation – Shared L2 – Manage one firewall per VM • Potential use of GRE tunnels to create single L2 overlay • GRE in tech preview for site to site VPN.
- 23. L3 isolation with distributed firewalls Tenant 10.1.0.2 Public Public IP address 1 VM 1 Internet 65.37.141.11 65.37.141.24 65.37.141.36 10.1.0.1 Pod 1 L2 Tenant 10.1.0.3 65.37.141.80 Switch 2 VM 1 Tenant 10.1.0.4 1 VM 2 L3 Core Pod 2 L2 Switch 10.1.8.1 … Tenant 10.1.16.12 10.1.16.1 2 VM 2 Load Pod 3 L2 Balancer Switch Tenant 2 VM 3 10.1.16.21 … Tenant 1 VM 3 10.1.16.47 Tenant 10.1.16.85 1 VM 4 Slide from Chiradeep Vittal
- 24. A customizable GUI AJAX + API
- 25. A very extensive API
- 26. API • Not really REST • A set of methods available over http(s) • Unauthenticated on integration port • Authenticated on 8080 using Access and Secret Key • Python/Ruby clients available • Internal Marvin client • CLI under development • Other clouds client support the API
- 27. Making API calls Can be authenticated or not. HTTP call. User Keys can be generated via the GUI Base url: http://<manager-host>:8080/client/api?…. def make_request(requests, secretKey): request = zip(requests.keys(), requests.values()) request.sort(key=lambda x: str.lower(x[0])) requestUrl = "&".join(["=".join([r[0], urllib.quote_plus(str(r[1]))]) for r in request]) hashStr = "&".join(["=".join([str.lower(r[0]), str.lower(urllib.quote_plus(str(r[1]))).replace("+", "%20")]) for r in request]) sig = urllib.quote_plus(base64.encodestring(hmac.new(secretKey, hashStr, hashlib.sha1).digest()).strip()) print "Signature: %s"%sig requestUrl += "&signature=%s"%sig print requestUrl if __name__ == '__main__': requests = { "apiKey": "BRZ5j4E8O4di2MZWnQsYBLThCrTGO-LGeZaMjsnvelkHuY5P8FdTnluNZTDQhCUy- wqeJzk8EAc_NbcZxTF_FA", "response" : "json", "command" : "listZones" } secretKey = "bFlx2llt3OmM4AiHzfwV1ZbuJ5tsv6hAx6IeM32CkM-obCA77BRwBr3_yQ0bO1-kdZyfD3-lY6khsXCx18n3Mw" make_request(requests, secretKey)
- 28. EC2 /S3 compatibility • Significant development work happening to make Cloudstack highly compatible with EC2 /S3 API. • http://wiki.cloudstack.org/display/RelOps/EC2+API +support+in+CloudStack • http://www.slideshare.net/sebastiengoasguen/clou dstack-ec2-configuration • Euca tools, boto etc…should work with cloudstack.
- 29. Enabling EC2 and S3 • Via the GUI • Via API call on integration API port 8096 http://localhost:8096/client/api? command=updateConfiguration&name=enable.s3.api&value=true http://localhost:8096/client/api? command=updateConfiguration&name=enable.ec2.api&value=true
- 30. Highly Scalable • See: – http://www.slideshare.net/cloudstack/scalability- 12819428 – From Alex Huang. 10k “resources” managed per Mgt server. 30k resources with 30k VM in simulation. • Management server can be setup in a multi- node configuration with a load-balancer and replicated MySQL.
- 31. SandBox: DevCloud • A Virtual box appliance packaged to provide a working CloudStack environment. • Aimed at developers but has other use cases: – Xen PV hosts gives nested virtualization – Local EC2/S3 Cloud on your laptop – Networking experiments ? • http://wiki.cloudstack.org/display/comm/Dev Cloud
- 32. DevCloud: self-contained CloudStack runs in the appliance
- 33. Testing “4.0” code in DevCloud • Deploy new CloudStack code in self-contained DevCloud: • mvn –P deps • ant rdeploy • ant rdeploydb – Wipes database of mgt server, you will need to reconfigure the “data center” • ant rdebug
- 34. DevCloud: as Host Run CloudStack on local machine Use DevCloud to setup hosts
- 35. DevCloud to test 4.1 branch • mvn -P developer clean • mvn -P developer install • mvn -P developer -pl developer –Ddeploydb • mvn -P developer -pl tools/devcloud – Ddeploydb • mvn -pl :cloud-client-ui jetty:run Configure infrastructure: • mvn -P developer,deploysvr -pl tools/devcloud -Ddeploysvr
- 36. Testing Framework –for the PyUG [environment] • Marvin is a Python dns=10.147.28.6 mshost=10.147.39.69 based framework to run mysql.host=10.147.39.69 tests against a [cloudstack] private.gateway=10.147.40.1 CloudStack install private.pod.startip=10.147.41.121 private.pod.endip=10.147.41.160 • Could be used as a private.netmask=255.255.254.0 public.gateway=10.147.40.1 simulator of a public.vlan.startip=10.147.41.162 datacenter public.vlan.endip=10.147.41.200 public.netmask=255.255.254.0 • Used to configure an hypervisor=XenServer host=10.147.40.10 infrastructure on a mgt host.password=password #storage pools server primary.pool=nfs://10.147.28.7:/expo rt/home/automation/sadhu/primary secondary.pool=nfs://10.147.28.6:/ex port/home/automation/sadhu/secondary
- 37. Opportunities for Java developers • Improve Maven build • Move to Spring framework • True REST API • Plugin framework with better componentization – Cocoon ? • Junit tests • Support for Hadoop as storage backend • …
- 38. Info • Apache incubator project • http://www.cloudstack.org • #cloudstack on irc.freenode.net • @cloudstack on Twitter • http://www.slideshare.net/cloudstack • http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !
Editor's Notes
- #16: https://cwiki.apache.org/CLOUDSTACK/building-with-maven.html
- #24: As vms get created and destroyed, CloudStack has to ensure the configuration of the host-based firewalls (iptables) is consistent with the security group rules programmed by the cloud user
- #26: http://download.cloud.com/releases/3.0.3/api_3.0.3/TOC_Root_Admin.html
- #36: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+devcloud+environment+setup
- #37: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Testing+with+Python