Using BigQuery as a main Big Data solution
Download as PPTX, PDF•0 likes•1,642 views
Nikolay Novozhilov of Wego.com introduces BigQuery, Google's service for interactive analysis of massive datasets, as Wego's main Big Data solution. Wego was founded in 2005 in Singapore as Asia Pacific and the Middle East's leading flight/hotel metasearch engine. BigQuery allows querying of billions of rows in seconds, uses a SQL-style query syntax, and pays only for resources used. Novozhilov discusses replacing MySQL with BigQuery and addresses concerns about data security, pricing, and open-source alternatives.
Using BigQuery as a main Big Data solution
- 1. Nikolay Novozhilov Wego.com Using BigQuery as a main Big Data solution
- 2. About Wego Wego.com is Asia Pacific and the Middle East’s leading flight/hotel metasearch engine used by millions of travelers. Wego was founded in 2005 in Singapore
- 3. Introducing BigQuery Service for interactive analysis of massive datasets (TBs) Query billions of rows: seconds to write, seconds to return Uses a SQL-style query syntax It's a service, accessed by a RESTful API Pay only for what you use Based on internal Google tool - Dremel Column oriented, append only…
- 4. Data architecture in Wego ...
- 5. Why did we do it? MySQL “Zoo” BigQuery
- 6. Why Hadoop is more popular?
- 7. My collection of concerns Your data goes to cloud Not open-source, Google can stop the service “Strange” pricing model Hadoop is trending, has bigger community Append only database ???
- 8. Costs: storage + cost per query Same fallacy again: “I want to launch a mom@pop – let’s buy a building” “I want to build a site – let’s by servers” “I want big data – let’s build a data-warehouse” Usual concerns: No realistic estimate upfront “Fear of running a query”
- 10. Append only… Slowly changing dimensions: daily re-load from MySQL daily upload from MySQL, keeping history Absolutely necessary updates: do you really need it? BigQuery allows to save query to initial table: Your table Query
- 11. Actually useful - “Discovery mode”
- 12. Actually useful Huge joins REGEXT_MATCH(), … Rich SQL - window functions Nested data
- 13. My answer
- 14. What is Big Data revolution? There is no difference between big data and small data anymore
- 16. “Yes, Sir, I tired to build an ROI case for our BI project - but I couldn’t access any reliable data!” TimoElliott.com
Editor's Notes
- #7: https://www.google.com.sg/trends/explore#q=BigQuery%2C%20Amazon%20Redshift%2C%20Apache%20Spark%2C%20ElasticSearch%2C%20Hadoop&cmpt=q&tz=Etc%2FGMT-8
- #14: https://www.google.com.sg/trends/explore#q=BigQuery%2C%20Amazon%20Redshift%2C%20Apache%20Spark%2C%20ElasticSearch%2C%20Hadoop&cmpt=q&tz=Etc%2FGMT-8