How Thermo Fisher Is Reducing Mass Spectrometry Experiment Times from Days to Minutes w/ MongoDB Atlas on AWS

Editor's Notes

• #3: ThermoFisher is the biggest company you’ve never heard about, we strive to be the world leader in serving science. We have 50,000 employees around the world. Our goal is to make the world healthier, cleaner and safer.
• #4: One of the products we make is a Mass Spectrometer. At the core of the instrument is ping-pong ball size metal cylinder called an Orbitrab. Which spins ionized molecules around for distances of several kilometers in a fraction of a second and measures their masses very accurately. It turns out there are quite a few applications for this capability.
• #5: ThermoFisher Mass Spectrometry instruments are used to detect Pollutants, if it is bad for you, our instruments will detect it. One of our customers is the Karolinska institute in Sweden, (this is the same university responsible for giving out Nobel prizes) and they processes 100k samples per year serving all of Sweden. Each of their high resolution instruments produces 100TB data per year.
• #6: For me, making the world a cleaner, safer place is personally meaningful. My son Landon was born with a Cleft lip and Pallet which is caused at least in part by exposure of the baby at a very early age (pea size) to some environmental condition: mercury, lead, a volatile organic. So preventing other children from being born with birth with defects and having safe and healthy lives is one thing that motivates me to come to work every day.
• #7: The next mission to mars in 2020 will carry a mass spec known as the Mars Organic Molecule Analyzer, or MOMA, which contains a design based on a ThermoFisher Linear Ion Trap Mass Spectrometer. Mars rover is not running MongoDB, but maybe as the NASA trend continues for using commercial products and Thermo increasingly adapts MongoDB, MongoDB will ship on a Mars Rover some day. You definitely couldn’t run DynamoDB on the mars rover, but you could run Mongo. ---- http://science.gsfc.nasa.gov/sed/bio/veronica.t.pinnick https://ep70.eventpilot.us/web/planner.php?id=ASMS16 Mars Organic Molecule Analyzer (MOMA) Mass Spectrometer: Performance Testing in GC-MS and LD-MS Modes of Operation
• #8: Our mass spectrometers are used in major sporting events to ensure an even playing field by detecting banned performance enhancing drugs. [optional] If an athlete is using synthetic Testosterone, the instruments are sufficiently sensitive and the analytical techniques sufficiently advanced to detect the difference between synthetic and natural testosterone. [extra] We have a marketing contract with CBS for any CSI TV shoes they use ThermoFisher equipment. [reference] http://www.nbcnews.com/storyline/2016-rio-summer-olympics/rio-olympics-top-anti-doping-scientist-cheats-will-probably-be-n573531
• #9: So this is what beer looks like in a mass spec. This is 100 samples of various types of beer. Each one of the variations in these peaks represents the unique flavonoids that make a product unique and give it a distinct smell and flavor. Our mass spectrometers are used for product authenticity studies.
• #10: Any MythBuster fans out there? Adam Savage actually spoke at the keynote of MongoDB world 2016 in New York, so that is why I am a Mongo fan, never mind the technical merits. In 2009 The Mythbusters Adam and Jamie use ThermoFisher Mass Spectrometer to determine if soda cans have rat pee on them. Really great episode, just search for “Rat Pee Soda”. In the experiment, they take 1000 soda cans and let rats run and pee all over them. And then take soda cans from local convenience stores and compare the two sets of cans using a black light. Using the black light, both sets look similar. Organic material glowing under the black light. However, when they take the rat pee cans and the convenience store cans to the Stanford analytical lab, the mass spectrometer is able to conclusively determine that no rat pee is found on the convenience store cans. [reference] Episode 135 http://www.dailymotion.com/video/x2n9enp (Starting at minute 7:30 Jamie and Adam visit Stanford lab and use Thermo Mass Specs)
• #11: Jamie Says quote “These Mass Spectrometers are extremely accurate, they can detect down to a femptomole, and if it says they aren’t in there, its not in there.” Adam was very relieved by this result and drank a soda.
• #13: To keep things interesting, I am going to do a live demo. This is always a risky proposition when trying to remote monitor a complex instrument that is more expensive than my house using a network that is potentially unpredictable. Let me focus on one of our application that just rolled out to production called “Instrument Connect” built using Mongo Atlas. “Instrument Connect” allows our customers to connect their mass spectrometers to the ThermoFisher cloud built on AWS. Customers can monitor instrument status from anywhere in the world and receive notification of any errors that occur. Instrument data is streamed up to the cloud where it can take advantage of the incredible processing power of the AWS cloud and users from around the world can collaborate on the experiments and results. The database which stores instrument status is MongoDB Atlas. We also built a prototype integration with Amazon Alexa allowing us to control the instrument with voice commands. [Demo outline] Open MS Instrument connect dashboard. Open Atlas Dashboard
• #14: This is the mass spec we will be remote monitoring. I didn’t have the budget to bring the instrument with me on stage so I’ll use remote desktop. Humor: Apparently this is the only shirt I own.
• #16: ThermoFisher is increasingly using MongoDB in its applications.
• #18: Mass Spectrometers have become so sensitive that they can measure the mass of a molecule down to the electron. This results in a huge amount of data.
• #19: Rich query language includes partial document updates,
• #20: MongoDB can store many types of data. Using MongoDB allows us to simplify our infrastructure. It also allows us to use a single set of tools for managing our data and our applications.
• #21: Now that MongoDB supports join operations, we can store both relational and document data in the same database. This greatly expands the type of application that can be built on MongoDB and simplifies our deployment since we only have one database rather than two.
• #23: MongoDB has climbed to the number 4 slot on db-engines ranking of most popular databases. This is based on metrics including job postings, stack overflow questions and google searches. Mongo is only behind Oracle, MySql, and SqlServer. Oracle which was first released in 1979, Sql Server in 1989, MySql in 1999 and MongoDB in 2009. Remarkable that MongoDB has made up so much ground on relational database technology which is 40 years old and doesn’t show any sign of slowing down.
• #24: Let me talk for a moment about some performance, scalability and cost comparisons that we did with MySql vs. MongoDB We apply the same scientific rigor as our customers when making a decision on which database to use.
• #25: (remove fro AWS)
• #26: MySQL not optimized: 21 lines MySQL optimized: 40 lines MongoDB: 1 line TODO: run test with larger data set.
• #27: MySQL not optimized: 21 lines MySQL optimized: 40 lines MongoDB: 1 line If I were to reduce my presentation to one slide, this would be that slide. This is a staggeringly awesome improvement in developer productivity.
• #28: MySQL not optimized: 21 lines MySQL optimized: 40 lines MongoDB: 1 line
• #29: Similar number of lines of code and performance. SQL Injection: Nice advantage of MongoDB is that the queries are strongly typed and no chance of SQL injection. After all these years SQL injection is still the number one security threat. TODO: measure performance
• #30: The application used in the major sporting event in summer Rio sporting event - TraceFinder switched from XML and SQLite to MongoDB. We could probably reduce this even further, but there is a dramatic decrease in cyclomatic complexity. TODO: measure cyclomatic complexity.
• #31: Here I am contrasting DynamoDB and MongoDB, I think that as usual the answer to which database is best for your application is “It depends”, but here is some information to help you make that choice for your application. Both of these databases are very good and I think both will continue to grow in popularity at a much faster rate than relational databases. Like me product manager says, I don’t need better decisions, I need better choices. And as a customer it is great to have multiple good database choices. The most obvious difference is that Dynamo is an AWS-only service and Mongo runs anywhere. Rich query language: With MongoDB, I can answer questions like which instruments had the highest utilization last month or what is the pump pressure were I see my pumps begin to fail. And I can find this out without writing any code using ad-hoc queries. With MongoDB, I can take advantage of rich features like joins, document validation, strongly typed queries, decimal data type, views, graph queries, and grouping, the aggregation pipeline, map-reduce, native spark connector, etc. According to db-engines, MongoDB and DynamoDB are the two top rated database in the category of Document DBs. MongoDB has a score of 325 and DyamoDB a score of 29. Native Integration with AWS: For example, if you want to take advantage of the native triggers available to execute a Lambda statement
• #32: Please don’t interpret this slide as you should always use MongoDB over S3. That would not be wise. S3 would far out perform MongoDB in other scenarios. In this particular case, MongoDB is a much better choice. This measurement was taken by running C# code from EC2 instance in AWS US-East region. The title of this slide might strike you as odd, comparing S3 with MongoDB. S3 is an powerful AWS service which can be used to store multi gigabyte files and tiny JSON objects. It is a key-value store but by carefully selecting keys you can use S3 like a simple database with tables and rows a set of S3 objects with the same key prefix can function like a database table, the advantage is that you have a very inexpensive, serverless, highly available database. But as your application gets more complex you miss out on the rich query capabilities of a full relational or document database. For our Real-time chromatogram we realized a couple orders of magnitude in savings in network and CPU consumption on our application servers by not having to download the entire S3 object and filter it down, we were able to do this instead on the database. [Reference] Performance measurement code: "C:\_git\CloudAgent\srcapi\Ironclad.Bootstrap\Repo\RealtimeChroDalBootstrap.cs" [Note] Serialzed JSON to S3 using Newtonsoft to S3 which is 20% larger objects compared with Mongo Bson. (storage on disk is even more of a contrast)
• #33: Please don’t interpret this slide as you should always use MongoDB over S3. That would not be wise. S3 would far out perform MongoDB in other scenarios. In this particular case, MongoDB is a much better choice. This measurement was taken by running C# code from EC2 instance in AWS US-East region. The title of this slide might strike you as odd, comparing S3 with MongoDB. S3 is an powerful AWS service which can be used to store multi gigabyte files and tiny JSON objects. It is a key-value store but by carefully selecting keys you can use S3 like a simple database with tables and rows a set of S3 objects with the same key prefix can function like a database table, the advantage is that you have a very inexpensive, serverless, highly available database. But as your application gets more complex you miss out on the rich query capabilities of a full relational or document database. For our Real-time chromatogram we realized a couple orders of magnitude in savings in network and CPU consumption on our application servers by not having to download the entire S3 object and filter it down, we were able to do this instead on the database. [Reference] Performance measurement code: "C:\_git\CloudAgent\srcapi\Ironclad.Bootstrap\Repo\RealtimeChroDalBootstrap.cs" [Note] Serialzed JSON to S3 using Newtonsoft to S3 which is 20% larger objects compared with Mongo Bson. (storage on disk is even more of a contrast)
• #35: With all the time we are saving writing and optimizing data layer code, we are able to invest in improving our algorithms, improving the user experience, and improving the processing infrastructure.
• #37: We have used MongoDB in a single server configuration, but we did not have expertise in cluster management, and don’t necessarily want to. When Atlas was announced in July of this year, we immediately jumped on board, ease of deploying MongoDB was the one thing holding us back. With a weekend of work I switched from Dynamo to MongoDB in one day. Switching databases two months before going to production is not something I necessarily recommend. On Monday I asked my boss, did you notice anything different about the application, any downtime, he said now. I told him that I switched the database to Mongo, and he was not enthusiastic Switching has turned out to be a great decision, and we have other applications looking to make the same switch. Robustness: We were storing some of our data in DynamoDB and some in S3 because Dynamo is expensive. But you can do partial updates of S3 documents we since moved this data to MongoDB have improved the robustness and performance significantly. We had a significant number of outages on Dynamo because we didn’t have sufficient throughput on our Dynamo tables, due to short spikes in traffic Rate of development: Adding a new collection is much easier than adding a table in Dynamo. I don’t have to write a cloud formation script each time I want to provision a new table. I don’t have to provision read/write capacity for each table of the database independently. Data analytics: Writing ad-hoc queries to answer questions like “give me a count of instruments per user”, or “what are my most active instruments” was not possible with DynamoDB because it doesn’t (didn’t) have a standalone query language or IDE. Ability to run outside cloud as well as inside the cloud