Node.js in a heterogeneous system
Download as PPTX, PDF0 likes1,649 views
The document outlines the use of Node.js within a heterogeneous system for big data analytics, emphasizing its fast performance and compatibility with AWS. It covers project structure using Maven and NPM, testing frameworks, and the deployment process managed by orchestration tools. Additionally, it includes details on setting up a test environment, build processes, and continuous integration practices.
![Test Env How-to
runtest.sh
#!/bin/bash
mocha $(find test -name '*.js') -R tap |
tee test/testresults.tap; (exit ${PIPESTATUS[0]})
The whole command runs in a single line. Wrapped here for display purpose](https://image.slidesharecdn.com/node-131130105549-phpapp01/85/Node-js-in-a-heterogeneous-system-25-320.jpg)
Node.js in a heterogeneous system
- 1. Node.js in a heterogeneous system Alexey Migutsky : Senior Software Developer @ Lohika
- 2. What do we do?
- 3. What do we do? Ubix.io PaaS for BigData analytics in cloud environments.
- 4. What do we use?
- 5. Scala Python What do we use? Bash Javascript
- 6. BDAS (Spark + Shark) AWS Black Voodoo Magic™ What do we use? Velocipede™ Brains
- 7. What’s next?
- 8. Node & Maven Jenkins & Node testing What’s next? Velocipede™ CI
- 9. ?
- 10. Why Node.js?
- 11. Why Node.js? Fast and easy Easy websockets support Plays nice with AWS We use it as a web-tier orchestrator
- 12. Node & Maven
- 13. Node & Maven We use maven (pom.xml) We use npm (packege.json) We have node+maven projects!
- 16. Project structure + Subprojects are maven modules + Subprojects use common code base + Easy « build process » + Easy deployments - Uber root package.json - Mangled dependencies
- 17. Custom modules
- 18. Custom modules Live in root /node_modules right now Planning to move to separate git repos Tested with subprojects
- 19. Tests
- 20. Tests Mocha + Sinon + Chai Sinon-chai + mocha-sinon Chai-as-promised Rewired Jenkins TAP plugin
- 21. Test example describe('##getConfiguration', function () { it('should return promise and resolve with data ', function (done) { var promise = sysStore.getConfiguration('somekey'); promise.should.eventually.become({key: 'somekey'}) .then(function () { mockDb.getItem.should.be.calledWithMatch( mockTables.system, {key: sinon.match.string} ); }) .should.notify(done); }); });
- 22. Test Env How-to
- 23. Test Env How-to Goals: - run from root using mvn test run from sub using npm test human-readable result logs result >> in console and file Jenkins support http://www.mindmeister.com/346337860
- 24. Test Env How-to package.json "devDependencies": { "mocha": "*", "sinon": "1.6.*", "chai": "*", "sinon-chai": "*", "mocha-sinon": "*", "chai-as-promised" : "*", "rewire" : "*" }, "scripts" : { "test" : "./runtest.sh" }
- 25. Test Env How-to runtest.sh #!/bin/bash mocha $(find test -name '*.js') -R tap | tee test/testresults.tap; (exit ${PIPESTATUS[0]}) The whole command runs in a single line. Wrapped here for display purpose
- 26. Test Env How-to pom.xml (maven-antrun-plugin) <execution> <id>test-nodejs</id> <phase>test</phase> <configuration> <tasks name="Run mocha tests"> <exec dir="${basedir}” executable="npm” failonerror="true"> <arg value="test"/> </exec> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution>
- 27. Build process
- 28. Build process - No npm install on build server (too slooow) - Fixed deps versions (update by-hand) - All deployable deps are checked in! - Build phase only copies deps in subprojects http://www.futurealoof.com/posts/nodemodules-in-git.html
- 29. Deployment
- 30. Deployment - Builds are stored at S3 - Copied to instances by orchestration tools - Managed by upstart
- 31. Deployment upstart (server.conf) #this should be put in /etc/init/server.conf on Ubuntu description ”Node.js server" author "Alexey Migutsky" # used to be: start on startup # until we found some mounts weren't ready yet while booting: start on started mountall stop on shutdown # Automatically Respawn: respawn respawn limit 20 5 # Max open files are @ 1024 by default. Bit few. limit nofile 32768 32768 script # Not sure why $HOME is needed, but we found that it is: export HOME="/root” exec /usr/local/bin/node /usr/local/server/app.js >> /var/log/server-err.log 2>&1 end script post-start script end script http://kvz.io/blog/2009/12/15/run-nodejs-as-a-service-on-ubuntu-karmic/
- 32. Deployment hint Have a look at http://pm2.io/
- 35. « hack some node.js » npm test
- 38. build & test
- 43. That’s it!