Quality, quantity, web and semantics
2 likes1,759 views
How is organized data used by some web players having not the best intentions? How can tools that try to help individual authors be subverted by spammers? Also, how does Zemanta work and why are we interested in this topic.
More Related Content
What's hot (13)
Viewers also liked (6)
Similar to Quality, quantity, web and semantics (20)
More from Andraz Tori (8)
Recently uploaded (20)
Quality, quantity, web and semantics
- 1. Quality to Quantity to Quality on the Web Andraž Tori, CTO at Zemanta @andraz
- 2. Topics - a bit about Zemanta - how advanced “data tools” and spammers interact
- 3. We are all trying to organize the web
- 4. Making it right, making it useful and linked
- 7. Not so long time ago, in a city not far away...
- 9. are trying to do the opposite
- 10. trying to disorganize it, make it confusing, and to profit from that
- 12. using the tools we have built!
- 14. Their motives are not sinster (mostly)
- 15. it is about profit
- 16. Profit - publish as much content as possible - quality is not (that) important - get traffic or high page ranking for certain terms - sell clicks, links or whole “fully built” sites to the highest bidder - users and search engines are necessary evil to be tricked as cheaply as possible
- 18. So, why do I care?
- 19. Job opening You will get a spreadsheet with 180 blog url’s and logins. You will log into each blog and schedule 2 posts per week ... You will spice up every post with images and/or related links within the content, using a Wordpress plugin called Zemanta https://www.odesk.com/jobs/Wordpress-Blog-Poster_~~c8c04549b8e6b600
- 20. And why might you care? - the organized information is great tool for those that try to disorganize it - they are poisoning “our web”, including twitter, facebook - and it's hard to see in the fog they are causing - it is just matter of time when they start poisioning linked data too
- 22. What do we do at
- 23. - is a “personal writing assistant” - suggesting content while you write (your blog) - analyzing your text - connecting it with background knowledge, other stories on the web, images - you choose what suggestions to include - to make your writing more informative, vivid and useful
- 27. Opening up the hood
- 30. the reality
- 32. How it works Content suggestions Plain text Semantic Analysis (article) search Linked data RSS feeds
- 33. Main design goals - Input is meaningful chunk of text (not a keyword or a phrase) - Input is (semi) English language - Has to work across all domains in the open world - music, celebrities, finance, entertainment, politics, gardening, parenting, …
- 34. Analysis pipeline Known phrases Named Entity extraction Extraction (aho-corasick) Triple store Surface form features evaluation Statistical comparison to background knowledge Semantic coherence and hand-tuned heuristics etc. Disambiguated entities
- 35. Analysis pipeline Known phrases Named Entity extraction Extraction (aho-corasick) Categorization to Dmoz Triple store Surface form features evaluation Statistical comparison to background knowledge Semantic coherence and hand-tuned heuristics etc. Categories Ambigious named entities Disambiguated entities
- 36. Background knowledge - Data from Wikipedia, MusicBrainz, Freebase… and world wild web - Includes linguistical and semantical properties + unstructured data - Present in two forms: - in “original” custom built triple store on top of MySQL (150 GB) - processed into 7 GB optimized “memory mapped dump”
- 37. Background knowledge - 7M mined and linked up entities and concepts Triple store - 30M aliases - Refreshed about once a month - want to make it real-time - Input data quality is really important etc.
- 38. After analysis Text SOLR Related articles articles SOLR Images images
- 40. boost((( wiki_entities:Health insurance wiki_entities:Medical underwriting wiki_entities:United States wiki_entities:Affordable Care Act wiki_entities:Barack Obama wiki_entities:Lifetime (TV network) wiki_entities:Insurance wiki_entities:Preventive medicine wiki_entities:Child wiki_entities:Patient Protection and Affordable Care Act ) ^3.0) (text:zemhealthinsurq^0.68 text:health^0.62 text:premium^0.36 text:zeminsurcompaniq^0.56 text:increas^0.29 text:rate^0.27 text:zemhealthinsurcompaniq^0.35 text:zempreventcareq^0.26 text:medic^0.26 text:compani^0.23 text:obamacar^0.21 text:todai^0.21 text:polici^0.21 text:care^0.19 ) ^105.0 ((dmoz_categories: Top/Business/Financial_Services/Insurance/Agents_and_Marketers/Health dmoz_categories: Top/Business/Financial_Services/Insurance/Agents_and_Marketers/Health/United_States dmoz_categories: Top/Business/Financial_Services/Insurance/Agents_and_Marketers/Health/United_States /California) ^0.1), (1 - 0.2) * sqrt(1.0/(1.15E-8*float(1285185600000 - date(published_datetime) ms)+1.0)) + 0.2)
- 41. Solr - We adapted Solr for “query by document” - 52% precision (at 10) on internal evaluations - plain Lucene MLT comes to 44% - difference is from “bag of terms” approach over “bag of words” (terms coming from analysis step) - Our live index is 5M articles - Solr is really not optimized to handle 50 terms in a single query
- 42. Lucene plain “More Like This”
- 43. Metrics & tests - Every part of the system is being constantly evaluted - Precision/recall at 5 different points in the system - Mostly bi-weekly releases of new datasets and the engine
- 44. Overview - We do pretty deep processing to deliver simple user experience of “personal authoring assistant” - And everything is available over the web API - tagging - named entity recognition and disambiguation to Linked Open Data URIs
- 45. What API offers? • Tags Most used • Categories • Concepts and entities Most interesting • Related articles • Related images
- 48. Some API users
- 49. We are just one of the many people offering services based on large amounts of web data each spending man-years trying to organize their data, trying to offer best possible service
- 50. now back to the bad guys
- 52. Job opening You will get a spreadsheet with 180 blog url’s and logins. You will log into each blog and schedule 2 posts per week ... You will spice up every post with images and/or related links within the content, using a Wordpress plugin called Zemanta https://www.odesk.com/jobs/Wordpress-Blog-Poster_~~c8c04549b8e6b600
- 53. There's more than meets the eye
- 54. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 55. Warnings - I've seen no single system using the whole pipeline as described, however all parts were found in the wild - Examples used are from all kinds of sites – good, bad and ugly - I am not trying to imply that all of the steps in the diagram are bad, but they can be used by bad guys efficiently
- 56. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 57. Finding their keywords, niches - Domain expertise - Users like to install extensions and say “yes” - You observe referrers on sites you control - You buy the data on the black market
- 59. The sophisticated part of the market “Demand Media relies on a proprietary algorithm to help editors best determine what subjects their writers should tackle.” Factors: - Keyword competition - Revenue - Driving traffic to/from existing conent http://emediavitals.com/article/16/demand-media-s-content-assembly-line
- 60. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 61. Find / create content - Steal - Take from “open article directories” - Have your own “content assembly line” like Demand Media
- 63. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 64. Tһiѕ iѕ nοt the text you аre lookinɡ for.
- 65. Tһiѕ iѕ nοt the text you аre lookinɡ for.
- 66. Translate it to random language and back to English Übersetzen sie zufällig Sprache und wieder auf Englisch Language and translate it happen again in English Μεταφράστε αυτό σε δειγματοληπτικούς γλώσσα και πίσω στην αγγλική γλώσσα Translate this random language back to English Traduisez au langage aléatoire et revenir à l'anglais Translate to random language to English and back 它翻译成随机的语言和回英文 Translate it back into the English language and random
- 67. Covering their tracks - Trying to fool search engines or people? - Search engines are catching up - Google Translate API is being closed due to “abuse”? - The trend is “rewriting” by human editors, procured on the global market
- 69. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta, OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 70. Spammers say darndest things
- 72. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 74. Remixing linked data and spam - Currently mostly the good guys are using Linked Data - However, it's just too tempting to be left alone - Fully synthetic articles using factual information from linked data? – Using advanced tools to form proper natural language sentences and maybe even storyline?
- 75. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 76. Publish - On hosted third party platforms - eating their resources - Platforms have hard time killing spammers - Smaller ones don't necessarily have the incentive - If they remove spammer too fast, it is easier for spammer to probe the limits - Platforms use “kill with delay” - Spam detection is resource intensive
- 77. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 78. Valuable comments As I write this post, Zemanta is showing me 5 different articles that are related to my post. I could visit each one of these sites and reach out to the owner to see if they would be interested in linking to my post, or I could leave a valuable comment on the page and include a link back to my post. http://www.mainelyseo.com/zemanta-review-seo-link-building-with-the-zemanta-plugin/
- 79. - Guy in previous slide is honest and well- meaning - But what if you automate that via Amazon Mechanical Turk or oDesk?
- 80. Gather search terms Analyze → Find / create (extensions, logs, guess) what people search for? such content Pull additional content Use Zemanta or OpenCalais Cover your tracks from Freebase to add tags, images, links Publish Amazon Mechanical Turk Use Zemanta to find to post comments Profit? similar blogs and links back to your site
- 81. Profit? - sell ads - sell links - sell “fully developed site” - to the highest bidder
- 82. Search engines to the rescue? - Mahalo cut 10% of the staff the day after Google announced ranking changes - Demand Media's stock isn't doing that well anymore - However this is a never-ending story, we'll have co-evolution for foreseeable future
- 83. Ecosystem - Very sophisticated, large players - moving to more high quality content, video? - Small time operations - using more and more sophisticated tools available on the market cheaply (modern asymmetric warfare?) - Dark industry specifically building tools to poison the web and sell them to small time operators
- 84. Food for thought
- 85. Can we make spammers (and others) work for us, making linked data better? (think reCAPTCHA)
- 86. Could article directories be fruitfully used? eZineArticles.com, GoArticles.com, etc...
- 87. Find rewritten articles and use them as parallel corpus?
- 88. Could we use global workforce market more efficiently to get more linked data?
- 89. Thesis, antithesis, synthesis? http://xkcd.com/810/
- 91. Image sources http://www.flickr.com/photos/dzingeek/4587871752/ http://www.flickr.com/photos/25101572@N02/4393474025/ http://www.flickr.com/photos/billward/4740384434/ http://www.flickr.com/photos/jurvetson/542500748 http://www.flickr.com/photos/legofenris/4288913574 http://www.flickr.com/photos/ekilby/3733627940 http://www.flickr.com/photos/ekilby/3732799269/ http://www.flickr.com/photos/cipherswarm/38354452 http://xkcd.com/810/