Parallel Rendering of Webpages
Download as PPTX, PDF1 like817 views
This document presents an approach to parallel rendering of webpages using a cluster of single board computers (SBCs) and discusses the related hardware and software environment. It highlights the advantages of this method, including improved code structure, easier reuse, reduced development times, and scalability. Additionally, it details the architecture of the SBC cluster, communication components, and the necessary software stack for deployment.
Parallel Rendering of Webpages
- 1. Parallel Rendering of Webpages An approach based on Python and a Cluster of Single Board Computers (SBCs)
- 2. Introduction ▪ This presentation describes the hardware and software environment for experimenting with the Parallel Rendering of Webpages; ▪ The hardware consists of a cluster of single board computers (SBCs) which run a number of application servers that previously ran in the cloud (RHC and Pythonanywhere). Each application server was associated with a single application which was either available online or installed on a mobile device; ▪ The cluster described here has 50 cores plus 16 Epiphany cores. A web server runs on each of the 50 cores. Two servers, running on the Parallella, have access to the 16 Epiphany cores; ▪ The software for rendering webpages, consists of a single library - the Parallel Template Library. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 2
- 3. Advantages of the Parallel Rendering of Webpages ▪ An unexpected advantage is that designing applications to run on a cluster improved the structure of the final code. ▪ Writing reusable code is a lot easier and in line with the original philosophy behind UNIX (https://en.wikipedia.org/wiki/Unix_philosophy). ▪ The code can run on a single node so creating low cost products is still possible. ▪ Development times are reduced because software components are less complex and their deployment easier to manage. ▪ The system can be scaled and managed easily. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 3
- 4. The UNIX philosophy ▪ Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". ▪ Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input. ▪ Design and build software, even operating systems, to be tried early, ideally within weeks (days). Don't hesitate to throw away the clumsy parts and rebuild them. ▪ Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them. Doug McIlroy, E. N. Pinson, B. A. Tague (8 July 1978). "Unix Time-Sharing System: Foreword" . The Bell System Technical Journal. Bell Laboratories. pp. 1902–1903. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 4
- 6. Main System Components ▪ Architecture ▪ Communication ▪ Programming 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 6
- 7. Main System Components Network and Computer Hardware Architecture File Sharing and Cluster Control Cummuication Python libraries Programming 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 7
- 8. System Architecture Wifi/LAN Network Switch Parrallella SB 2 Computer SB n+1 SB 8 Network Switch Control Node 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 8
- 9. • Wifi router (ISP) • Switch (TP-Link) • Control Node • ASUS tinker board (Linaro Debian) • Single Board (SB) Computers • Raspberry pi (Raspbian) • Ordroid C1+ (Odrobian) • Odroid C2 (Ubuntu) • Orange pi pc (Armbian) • Banana pi (Bananian) • Nano pi M2, • Parallella 16 (Linaro Debian) System Architecture - components 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 9
- 10. System Architecture - Communication openssh Samba/NFS Python 2.7 Mico- web Frameworks 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 10
- 11. The Software stack Application PRW Libraries Bottle, Waitress, Socketserver and Jina2 The minimum number of Python packages needed to get started are Bottle, Waitress and Jinja2 System Architecture - software 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 11
- 12. Single Board Computers Computers < $100
- 13. Asus tinker board 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 13 • 2GB dual-channel LPDDR3 memory • Gigabit LAN and Bluetooth 4.0 + EDR connectivity • 802.11 b/g/n Wi-Fi • Four USB 2.0 ports • 40-pin internal header with 28 GPIO pins • Contact points for PWM and S/PDIF signals • 3.5mm audio jack connection • CSI port for camera connection • DSI port supporting HD resolution • HDMI 2.0 port with 4K-resolution support • MicroSD port supporting UHS-I card speed • Supports Debian OS with Kodi • 5V/2A Micro-USB power supply (not included)
- 14. Odroid c2 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 14 • Amlogic ARM® Cortex®-A53(ARMv8) 1.5Ghz quad core CPUs • Mali™-450 GPU (3 Pixel-processors + 2 Vertex shader processors) • 2Gbyte DDR3 SDRAM • Gigabit Ethernet • HDMI 2.0 4K/60Hz display • H.265 4K/60FPS and H.264 4K/30FPS capable VPU • 40pin GPIOs + 7pin I2S • eMMC5.0 HS400 Flash Storage slot • UHS-1 SDR50 MicroSD Card slot • USB 2.0 Host x 4, USB OTG x 1 (power + data capable) • Infrared(IR) Receiver • Ubuntu 16.04 or Android 5.1 Lollipop based on Kernel 3.14LTS
- 15. Raspberry pi 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 15 • SoC: Broadcom BCM2837 • CPU: 4× ARM Cortex-A53, 1.2GHz • GPU: Broadcom VideoCore IV • RAM: 1GB LPDDR2 (900 MHz) • Networking: 10/100 Ethernet, 2.4GHz 802.11n wireless • Bluetooth: Bluetooth 4.1 Classic, Bluetooth Low Energy • Storage: microSD • GPIO: 40-pin header, populated • Ports: HDMI, 3.5mm analogue audio-video jack, 4× USB 2.0, Ethernet, Camera Serial Interface (CSI), Display Serial Interface (DSI)
- 16. Banana pi 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 16 • SoC – Allwinner A31s quad core Cortex A7 @ 1.0 GHz with PowerVR SGX544MP2 GPU • System Memory – 1GB DDR3 • Storage – Micro SD slot up to 64GB • Connectivity – 10/100/1000 Ethernet, 802.11 b/g/n Wi- Fi (Realtek) • Video Output – HDMI, CVBS, and LVDS/RGB header • Audio Output – HDMI and 3.5mm stereo jack • Camera – CSI connector • USB – 2x USB 2.0 host ports + micro USB port (power only) • Debugging – UART pins for serial console • Expansion Header – 40-pin R-Pi “somewhat” compatible header with 2×13 UART, I2C bus, SPI bus, CAN bus, ADC, PWM, 3.3V, 5V, ground • Misc – IR Receiver, power, reboot and reset buttons. • Power Supply – 5V in via MicroUSB (DC in only)
- 17. Odroid c1+ 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 17 • Amlogic ARM® Cortex®-A5(ARMv7) 1.5Ghz quad core CPUs • Mali™-450 MP2 GPU (OpenGL ES 2.0/1.1 enabled for Linux and Android) • 1Gbyte DDR3 SDRAM • Gigabit Ethernet • 40pin GPIOs + 7pin I2S • eMMC4.5 HS200 Flash Storage slot • UHS-1 SDR50 MicroSD Card slot • USB 2.0 Host x 4, USB OTG x 1 (power + data capable) • Infrared(IR) Receiver • Ubuntu or Android OS
- 18. Orange pi pc 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 18 • SoC – Allwinner H3 quad core Cortex A7 @ 1.3 GHz with ARM Mali-400MP2 GPU up to 600 MHz • System Memory – 1GB DDR3 • Storage – 8GB eMMC flash + micro SD card slot • Video Output – HDMI with CEC, AV port • Audio I/O – HDMI, AV port, on-board microphone • Connectivity – 10/100M Ethernet, 802.11 b/g/n WiFi with external antenna • USB – 3x USB 2.0 host ports, 1x micro USB OTG port • Camera – CSI Interface • Expansions – 40-pin Raspberry Pi compatible header with 28 GPIOs, UART, I2C, SPI, PWM, CAN, I2S, SPDIF, LRADC, ADC, LINE-IN, FM-IN, and HP-IN • Debugging – 3-pin UART header for serial console • Misc – IR receiver; Power button; Power and status LEDs • Power Supply – 5V/2A via barrel jack (micro USB OTG cannot be used to power the board).
- 19. Nanopi M1 ▪ CPU: Allwinner H3, Quad-core [email protected] ▪ GPU: Mali400MP2@600MHz，Supports OpenGL ES2.0 ▪ DDR3 RAM:512MB/1GB ▪ Connectivity: 10/100M Ethernet ▪ Audio: 3.5mm audio jack/Via HDMI ▪ Microphone: Onboard microphone ▪ IR Receiver: Onboard IR receiver ▪ USB Host：Type A, USB 2.0 x 3 ▪ MicroSD Slot x 1 ▪ MicroUSB: for data transmission and power input, OTG ▪ Video Output: HDMI 1.4 1080P, CVBS ▪ DVP Camera Interface: 24pin, 0.5mm pitch FPC seat ▪ User Key: Power LED x 1, Reset x 1 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 19
- 20. Nanopi M3 ▪ SoC： Samsung S5P6818 Octa-Core Cortex-A53, 400M Hz - 1.4G Hz ▪ Power Management Unit： AXP228 PMU, it supports software power-off and wake-up. ▪ System Memory： 1GB 32bit DDR3 RAM ▪ Storage： 1 x MicroSD Card Socket ▪ Ethernet： Gbit Ethernet(RTL8211E) ▪ WiFi： 802.11b/g/n ▪ Bluetooth： 4.0 dual mode ▪ Antenna： Porcelain Antenna IPX Interface ▪ Audio： 3.5 mm audio jack / via HDMI ▪ I2S: 7pin, 2.54mm pitch pin-header ▪ Microphone: onboard Microphone ▪ USB Host： 4 x USB 2.0 Host, two type A ports and two 2.54 mm pitch pin-headers ▪ MicroUSB: 1 x MicroUSB 2.0 Client, Type A ▪ LCD Interface： 0.5mm pitch 45 pin FPC seat, full color RGB 8-8-8 ▪ HDMI: 1.4A Type A, 1080P ▪ LVDS: 20pin 2.00mm pitch pin-header ▪ DVP Camera： 0.5mm pitch 24 pin FPC seat ▪ User Key: K1 (power), Reset ▪ LED： 1 x power LED and 1 x status LED 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 20
- 21. Nanopc T3 ▪ SoC: Samsung S5P6818 Octa-Core Cortex-A53, 400M Hz - 1.4G Hz ▪ Power Management Unit: AXP228 PMU, it supports software power-off and wake-up. ▪ System Memory: 1GB/2GB 32bit DDR3 RAM ▪ Storage: 1 x SD Card Socket ▪ Ethernet: Gbit Ethernet(RTL8211E) ▪ WiFi: 802.11b/g/n ▪ Bluetooth: 4.0 dual mode ▪ Antenna: Porcelain Antenna IPX Interface ▪ eMMC: 8GB ▪ Video Input: DVP Camera/MIPI-CSI (two camera interfaces) ▪ Video Output: HDMI Type-A / LVDS / Parallel RGB-LCD / MIPI-DSI (four video output interfaces) ▪ Audio: 3.5 mm audio jack / via HDMI ▪ Microphone: onboard Microphone ▪ USB Host: 4 x USB 2.0 Host, two type A ports and two 2.54 mm pitch pin-headers ▪ MicroUSB: 1 x MicroUSB 2.0 Client, Type A ▪ LCD Interface: 0.5mm pitch 45 pin FPC seat, full color RGB 8-8-8 ▪ HDMI: 1.4A Type A, 1080P ▪ DVP Camera: 0.5mm pitch 24 pin FPC seat ▪ GPIO: 2.54 mm pitch 30 pin-header ▪ Serial Debug Port: 2.54mm pitch 4-pin-header ▪ User Key: K1 (power), Reset ▪ LED: 1 x power LED and 2 x GPIO LED 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 21
- 22. Parallella 16 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 22 • 18-core credit card sized computer • #1 in energy efficiency @ 5W • 16-core Epiphany RISC SOC • Zynq SOC (FPGA + ARM A9) • Gigabit Ethernet • 1GB SDRAM • Micro-SD storage • Up to 48 GPIO pins • HDMI, USB (optional)
- 24. Tinker OS A debian distribution driven by the Linaro arm ecosystem and downloadable fomr the ASUS tinker board product page https://www.asus.com/uk/Single-board-Computer/TINKER-BOARD/ 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 24
- 25. Ubuntu for Odroid c2 An Ubuntu Mate distribution downloadable from Hardkernel’s wiki: http://odroid.com/dokuwiki/doku.php?id=en:odroid-c2 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 25
- 26. Odrobian for c1+ ODROBIAN is a pure, clean and lightweight Debian community distribution built specifically for ODROID single board computers. The project is about getting the best experience out of Linux built upon Debian for ODROIDs, since there ain't any official images for this particular platform while it's mostly known for stability over ubuntu, the main goal is to represent a perfectly working OS with the least issues possible, hopefully. http://oph.mdrjr.net/odrobian/images/s805/ 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 26
- 27. Armbian for the Orange pi pc A lightweight Debian Jessie or Ubuntu Xenial based Linux distribution. https://www.armbian.com/orange-pi-pc/ 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 27
- 28. Bananian Bananian Linux is a pre-installed Debian 8 image optimized for Banana Pi/Pro. It uses the official Debian Jessie armhf repositories with a kernel and bootmanager (u-boot), customized for Banana Pi/Pro. A Debian 7/Wheezy image (Bananian 15.04) is also available and maintained. We support the Banana Pi (M1, M1+), Banana Pro and BPI-R1 with just one single image. A separate image for the Banana Pi M2 is also available. https://www.bananian.org/bananapi-m2 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 28
- 29. Parabuntu (linaro) An Ubuntu distribution driven by the Linaro arm ecosystem and downloadable from their company website: https://www.parallella.org/quick-start/ Parabuntu is the recommended Linux distribution which contains the firmware for Parallella boards. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 29
- 30. Parallel Rendering of Webpages Rendering Webpages
- 31. Parallel Template Library • The Parallel Template Library (PRTL) is based on a very simple idea – let the variable names to the work. Within a template, the node where the content for a particular page component is generated is specified by the variable name. Variable names within the template translate into a web address. For instance this code <h2>Crossword</h2> <div class="wrapper">{{realtime_P1_V_cp}}</div> tells the Parallel Template Library to obtain the content from: http://realtime:4001/cp • A variable is encoded in this way: <hostname><port>_V_<method> • For this example, all port addresses are offset by 4000 • The PTL is essentially a preprocessor which finds variables referencing remote content and then returns the fetched content. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 31
- 32. 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 32 Example Application – Python server code
- 33. Example Application - Jinja2 Template
- 34. Applications built with the Parallel Template Library Offline content generation
- 35. Offline media library and realtime teaching tools 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 35 • The offline media library runs on an SBC cluster or single SBC such as the raspberry pi and allows users to view content from Wikipedia, TED and Project Gutenberg without having an internet connection. • Users can also generate tests and word games based on viewed content. • The parallel Template Library was used to combine the functionality of the existing applications described above to create this app.
- 36. Summary A real-time tool for teachers and students alike – a synthesis of software and SBs
- 37. Developing apps to run on SBC clusters 5/10/2017Andrew Ritz - ©Langtech, Katowice, Poland 37 The parallel rendering of Webpages facilitates the creation of new applications from existing ones without having to write too much code. This is because the PTL allows new applications to be designed and created by combining together the functionality of a large number of dedicated web servers for the creation of new apps.