{"id":4616,"date":"2014-07-25T17:05:47","date_gmt":"2014-07-25T09:05:47","guid":{"rendered":"http:\/\/maziyar.me\/?p=4616"},"modified":"2016-03-14T16:37:49","modified_gmt":"2016-03-14T14:37:49","slug":"building-a-real-time-news-analytics","status":"publish","type":"post","link":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/","title":{"rendered":"Building a real-time news analytics"},"content":{"rendered":"<p>It is always interesting to find out what exactly is happening around the world as they are happening. This feature of knowing something as it happens is so called &#8220;realtime&#8221; or &#8220;near realtime&#8221; because of the network latency, delay for processing the data, delay for visualizing and streaming the data, etc.<\/p>\n<p>Now imagine you can monitor most international news agencies on Twitter in real time. What I have been developing for the last few months is an application that allows you to not only get important news and highlights around the world as they are happening but also persists the data for the window of 24 hours so you can always have the ability to read important news and highlights that have already happened.<\/p>\n<p><!--more--><\/p>\n<p>My study has shown that most news agencies publish their highlights and breaking news first in Twitter and then complete and develop the news as an article in their Website or they publish the article and share it on Twitter simultaneously.<\/p>\n<p>Twitter is famous for its real time feature and it kills the purpose to share something that happened days ago. Therefore, most news agencies try to publish their news and highlights on Twitter as soon as possible.<\/p>\n<p>I take advantage of this nature and will share some of the works that I have been doing by monitoring over 300 official news agencies on Twitter in real time.<\/p>\n<p>Some of the technologies that I use in the application architecture:<\/p>\n<p>Amazon Web Services (<a style=\"color: #006699;\" href=\"http:\/\/aws.amazon.com\" target=\"_blank\" rel=\"nofollow\">AWS<\/a>),\u00a0<a style=\"color: #006699;\" href=\"http:\/\/www.mongodb.org\/\" target=\"_blank\" rel=\"nofollow\">MongoDB<\/a>,\u00a0<a style=\"color: #006699;\" href=\"http:\/\/www.elasticsearch.org\" target=\"_blank\" rel=\"nofollow\">ElasticSearch<\/a>,\u00a0<a style=\"color: #006699;\" href=\"http:\/\/nodejs.org\/\" target=\"_blank\" rel=\"nofollow\">Node.js<\/a>, and\u00a0<a style=\"color: #006699;\" href=\"https:\/\/developer.apple.com\/\" target=\"_blank\" rel=\"nofollow\">Objective-C<\/a>\u00a0for mobile devices.<\/p>\n<p>For more information about the visualization, you can refer to my<a href=\"http:\/\/www.linkedin.com\/today\/post\/article\/20140606064859-142806667-building-your-own-search-engine-by-elastisearch\" target=\"_blank\">\u00a0post\u00a0on LinkedIn<\/a> .<\/p>\n<p>In this figure what we have is total number of news per 15 minutes. I have tried 5 minutes and 30 minutes but at the end I found out the 15 minutes window is a reasonable time frame for having the news in real time. For instance as it can be seen in figure below, there were 85 news in 15 minutes at that specific time.<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/8\/005\/06e\/16b\/2730198.jpg\" alt=\"\" \/><\/p>\n<p>Source of the tweets are always interesting to me. Although what we have here is just about news agencies and reporters, still has its own value to know what platform or device these people are more comfortable with.<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/7\/005\/06e\/16d\/01367bb.jpg\" alt=\"\" \/><\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/8\/005\/06e\/16d\/3108725.jpg\" alt=\"\" \/><\/p>\n<p>The other factor in each tweet (news or highlights), which it could be held as a metric of their importance to the community is the number of retweets. The application is smart enough to update the retweet number in near real-time but if you want to visualize how many RT has happened per 10 minutes you will get something like this:<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/6\/005\/06e\/16d\/3d772c2.jpg\" alt=\"\" \/><\/p>\n<p>Number of news that each agencies are publishing on their Twitter accounts is another interesting part here:<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/7\/005\/06e\/16e\/0d500fb.jpg\" alt=\"\" \/><\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/7\/005\/06e\/173\/117ef72.jpg\" alt=\"\" \/><\/p>\n<p>I did a simple topN on hashtags in these news to see a rough analytics of what they were saying:<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/7\/005\/06e\/172\/085d710.jpg\" alt=\"\" \/><\/p>\n<p>It is a rough analytics but still gives you some idea. For instance I had no idea what #takeoffjustlogo was until I saw it here and searched about it. I would do the topN on a small time frame to find out more about the trending hashtags in future but nonetheless.<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/5\/005\/06e\/173\/0eb3eb8.jpg\" alt=\"\" \/><\/p>\n<p>And last but not least, as it can be seen in this figure I did filter the data within a time frame that had most number of retweets to see what would have caused such a behavior:<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/7\/005\/06e\/175\/31bec73.jpg\" alt=\"\" \/><\/p>\n<p>And there it is:<\/p>\n<p><img decoding=\"async\" class=\"left\" src=\"http:\/\/m.c.lnkd.licdn.com\/mpr\/mpr\/p\/5\/005\/06e\/175\/3a52215.jpg\" alt=\"\" \/><\/p>\n<p>The iPhone app is available on App Store:\u00a0<a style=\"color: #006699;\" href=\"https:\/\/itunes.apple.com\/us\/app\/24hours-news\/id893024929?mt=8\" target=\"_blank\" rel=\"nofollow\">24hours News<\/a><\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It is always interesting to find out what exactly is happening around the world as they are happening. This feature of knowing something as it happens is so called &#8220;realtime&#8221; or &#8220;near realtime&#8221; because of the network latency, delay for processing the data, delay for visualizing and streaming the data, etc. Now imagine you can &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Building a real-time news analytics&#8221;<\/span><\/a><\/p>\n","protected":false},"author":65,"featured_media":5090,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[7],"tags":[12,14,18,19,23,36,38,42,43,44,46],"class_list":["post-4616","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-geeky","tag-aws","tag-bigdata","tag-ec2","tag-elasticsearch","tag-featured","tag-mongodb","tag-node-js","tag-redis","tag-search-engine","tag-socketio","tag-twitter"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Building a real-time news analytics<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building a real-time news analytics\" \/>\n<meta property=\"og:description\" content=\"It is always interesting to find out what exactly is happening around the world as they are happening. This feature of knowing something as it happens is so called &#8220;realtime&#8221; or &#8220;near realtime&#8221; because of the network latency, delay for processing the data, delay for visualizing and streaming the data, etc. Now imagine you can &hellip; Continue reading &quot;Building a real-time news analytics&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/\" \/>\n<meta property=\"og:site_name\" content=\"Maziyar PANAHI\" \/>\n<meta property=\"article:published_time\" content=\"2014-07-25T09:05:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-03-14T14:37:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png\" \/>\n\t<meta property=\"og:image:width\" content=\"2794\" \/>\n\t<meta property=\"og:image:height\" content=\"1452\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Maziyar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Maziyar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/\"},\"author\":{\"name\":\"Maziyar\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/#\\\/schema\\\/person\\\/02ae0118261368c80d32bef0b75f5faf\"},\"headline\":\"Building a real-time news analytics\",\"datePublished\":\"2014-07-25T09:05:47+00:00\",\"dateModified\":\"2016-03-14T14:37:49+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/\"},\"wordCount\":571,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/wp-content\\\/uploads\\\/sites\\\/5\\\/2015\\\/06\\\/Screenshot-2014-06-12-17.45.22.png\",\"keywords\":[\"AWS\",\"BigData\",\"EC2\",\"ElasticSearch\",\"featured\",\"MongoDB\",\"Node.js\",\"Redis\",\"Search Engine\",\"Socketio\",\"twitter\"],\"articleSection\":[\"Geeky\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/\",\"url\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/\",\"name\":\"Building a real-time news analytics\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/wp-content\\\/uploads\\\/sites\\\/5\\\/2015\\\/06\\\/Screenshot-2014-06-12-17.45.22.png\",\"datePublished\":\"2014-07-25T09:05:47+00:00\",\"dateModified\":\"2016-03-14T14:37:49+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/#\\\/schema\\\/person\\\/02ae0118261368c80d32bef0b75f5faf\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#primaryimage\",\"url\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/wp-content\\\/uploads\\\/sites\\\/5\\\/2015\\\/06\\\/Screenshot-2014-06-12-17.45.22.png\",\"contentUrl\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/wp-content\\\/uploads\\\/sites\\\/5\\\/2015\\\/06\\\/Screenshot-2014-06-12-17.45.22.png\",\"width\":2794,\"height\":1452},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/2014\\\/07\\\/25\\\/building-a-real-time-news-analytics\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building a real-time news analytics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/#website\",\"url\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/\",\"name\":\"Maziyar PANAHI\",\"description\":\"Generative AI Leader in Healthcare\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/#\\\/schema\\\/person\\\/02ae0118261368c80d32bef0b75f5faf\",\"name\":\"Maziyar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g\",\"caption\":\"Maziyar\"},\"description\":\"Big Data engineer\",\"sameAs\":[\"https:\\\/\\\/multivac.iscpif.fr\"],\"url\":\"https:\\\/\\\/iscpif.fr\\\/maziyar\\\/author\\\/maziyar-panahi\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Building a real-time news analytics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/","og_locale":"en_US","og_type":"article","og_title":"Building a real-time news analytics","og_description":"It is always interesting to find out what exactly is happening around the world as they are happening. This feature of knowing something as it happens is so called &#8220;realtime&#8221; or &#8220;near realtime&#8221; because of the network latency, delay for processing the data, delay for visualizing and streaming the data, etc. Now imagine you can &hellip; Continue reading \"Building a real-time news analytics\"","og_url":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/","og_site_name":"Maziyar PANAHI","article_published_time":"2014-07-25T09:05:47+00:00","article_modified_time":"2016-03-14T14:37:49+00:00","og_image":[{"width":2794,"height":1452,"url":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","type":"image\/png"}],"author":"Maziyar","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Maziyar","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#article","isPartOf":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/"},"author":{"name":"Maziyar","@id":"https:\/\/iscpif.fr\/maziyar\/#\/schema\/person\/02ae0118261368c80d32bef0b75f5faf"},"headline":"Building a real-time news analytics","datePublished":"2014-07-25T09:05:47+00:00","dateModified":"2016-03-14T14:37:49+00:00","mainEntityOfPage":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/"},"wordCount":571,"commentCount":0,"image":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#primaryimage"},"thumbnailUrl":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","keywords":["AWS","BigData","EC2","ElasticSearch","featured","MongoDB","Node.js","Redis","Search Engine","Socketio","twitter"],"articleSection":["Geeky"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/","url":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/","name":"Building a real-time news analytics","isPartOf":{"@id":"https:\/\/iscpif.fr\/maziyar\/#website"},"primaryImageOfPage":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#primaryimage"},"image":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#primaryimage"},"thumbnailUrl":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","datePublished":"2014-07-25T09:05:47+00:00","dateModified":"2016-03-14T14:37:49+00:00","author":{"@id":"https:\/\/iscpif.fr\/maziyar\/#\/schema\/person\/02ae0118261368c80d32bef0b75f5faf"},"breadcrumb":{"@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#primaryimage","url":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","contentUrl":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","width":2794,"height":1452},{"@type":"BreadcrumbList","@id":"https:\/\/iscpif.fr\/maziyar\/2014\/07\/25\/building-a-real-time-news-analytics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/iscpif.fr\/maziyar\/"},{"@type":"ListItem","position":2,"name":"Building a real-time news analytics"}]},{"@type":"WebSite","@id":"https:\/\/iscpif.fr\/maziyar\/#website","url":"https:\/\/iscpif.fr\/maziyar\/","name":"Maziyar PANAHI","description":"Generative AI Leader in Healthcare","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/iscpif.fr\/maziyar\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/iscpif.fr\/maziyar\/#\/schema\/person\/02ae0118261368c80d32bef0b75f5faf","name":"Maziyar","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/286798afc2f09c77b0ab1a0a60150f989d903d23d84c3eedfb8ebd1da500e5b1?s=96&d=mm&r=g","caption":"Maziyar"},"description":"Big Data engineer","sameAs":["https:\/\/multivac.iscpif.fr"],"url":"https:\/\/iscpif.fr\/maziyar\/author\/maziyar-panahi\/"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/Screenshot-2014-06-12-17.45.22.png","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6jMg3-1cs","jetpack-related-posts":[{"id":5278,"url":"https:\/\/iscpif.fr\/maziyar\/2015\/12\/03\/big-data-how-to-scale-from-zero-to-billions\/","url_meta":{"origin":4616,"position":0},"title":"Big Data: HOW TO SCALE FROM ZERO TO BILLIONS!","author":"Maziyar","date":"December 3, 2015","format":false,"excerpt":"How Big Data platform scaled from zero to billions of data within 6 months at ISCPIF (CNRS). This talk contains our use of Elasticsearch, MongoDB, Redis, RabbitMQ and scalable\/high available Web services built over Big Data architecture. This presentation was presented at Universit\u00e9 Paris-Sud, LAL, B\u00e2timent 200 organized by ARGOS.\u2026","rel":"","context":"In &quot;iscpif&quot;","block_context":{"text":"iscpif","link":"https:\/\/iscpif.fr\/maziyar\/category\/iscpif\/"},"img":{"alt_text":"real-time data analytics","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/12\/Screen-Shot-2015-12-03-at-12.23.21.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":4526,"url":"https:\/\/iscpif.fr\/maziyar\/2013\/11\/24\/source-of-tweets\/","url_meta":{"origin":4616,"position":1},"title":"Source of Tweets in France and United States","author":"Maziyar","date":"November 24, 2013","format":false,"excerpt":"So here is the thing: How people are sharing on Twitter around the world? What are the devices or services they usually use to share their check-ins, photos, videos, or updates on Twitter? This is a really simple analytics I did on a data that I've been gathering for almost\u2026","rel":"","context":"In &quot;Geeky&quot;","block_context":{"text":"Geeky","link":"https:\/\/iscpif.fr\/maziyar\/category\/geeky\/"},"img":{"alt_text":"US97","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/US97-640x237.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/US97-640x237.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/US97-640x237.png?resize=525%2C300 1.5x"},"classes":[]},{"id":4474,"url":"https:\/\/iscpif.fr\/maziyar\/2013\/09\/30\/european-conference-on-complex-systems-13\/","url_meta":{"origin":4616,"position":2},"title":"European Conference on Complex Systems 13","author":"Maziyar","date":"September 30, 2013","format":false,"excerpt":"I was at Barcelona this September to share the research project that we've done with other scientists and researchers at ECCS'13. The track we were on was Big Data in Complex Systems and the title of my talk was: Real-time Twitter Processing and\u00a0Visualization\u00a0on Geo-location Map by Using Cloud-based Platform.","rel":"","context":"In &quot;Events&quot;","block_context":{"text":"Events","link":"https:\/\/iscpif.fr\/maziyar\/category\/events\/"},"img":{"alt_text":"Before my presentation at ECCS'13 on Big Data on Complex Systems","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/1236000_10201058202994859_485889486_n-480x640.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":5209,"url":"https:\/\/iscpif.fr\/maziyar\/2015\/11\/15\/paris-before-and-after-terrorist-attacks\/","url_meta":{"origin":4616,"position":3},"title":"Paris before and during terrorist attacks","author":"Maziyar","date":"November 15, 2015","format":false,"excerpt":"\u00a0 Night of terror The data from Twitter shows\u00a0some upsetting stats\u00a0about Paris terrorist attacks on 13 November. As it can be seen, the query for terrorist attacks has\u00a0no result before 22h30. Unfortunately this shows even without any knowledge of the event itself (media or news), it is indeed possible to\u2026","rel":"","context":"In &quot;Daily&quot;","block_context":{"text":"Daily","link":"https:\/\/iscpif.fr\/maziyar\/category\/daily\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/Screen-Shot-2015-11-16-at-16.20.54.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":5231,"url":"https:\/\/iscpif.fr\/maziyar\/2015\/11\/28\/big-data-what-i-am-thankful-this-thanksgiving\/","url_meta":{"origin":4616,"position":4},"title":"Big Data: What I am thankful this Thanksgiving!","author":"Maziyar","date":"November 28, 2015","format":false,"excerpt":"Big Data platform at ISCPIF: This list represents what I am thankful to\u00a0this Thanksgiving Elasticsearch You know for search! This is the heart and the soul of our discovery and curiosity. The first step of every data analytics is finding the right data. Behind every great data discovery there must\u2026","rel":"","context":"In &quot;iscpif&quot;","block_context":{"text":"iscpif","link":"https:\/\/iscpif.fr\/maziyar\/category\/iscpif\/"},"img":{"alt_text":"big data at iscpif","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/11\/header.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":4697,"url":"https:\/\/iscpif.fr\/maziyar\/2015\/06\/16\/multivac-open-data-platform-for-scientific-community\/","url_meta":{"origin":4616,"position":5},"title":"MULTIVAC: open data platform for scientific community\u00a0","author":"Maziyar","date":"June 16, 2015","format":false,"excerpt":"ISCPIF\u00a0at innovatives SHS #bigdata #multivac #opendata #gargantext\u00a0#openmole\u00a0#InnovativesCNRS #textmining #visualization","rel":"","context":"In &quot;iscpif&quot;","block_context":{"text":"iscpif","link":"https:\/\/iscpif.fr\/maziyar\/category\/iscpif\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/iscpif.fr\/maziyar\/wp-content\/uploads\/sites\/5\/2015\/06\/IMG_0613.jpg?resize=1400%2C800&ssl=1 4x"},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/posts\/4616","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/users\/65"}],"replies":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/comments?post=4616"}],"version-history":[{"count":3,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/posts\/4616\/revisions"}],"predecessor-version":[{"id":5293,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/posts\/4616\/revisions\/5293"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/media\/5090"}],"wp:attachment":[{"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/media?parent=4616"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/categories?post=4616"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/iscpif.fr\/maziyar\/wp-json\/wp\/v2\/tags?post=4616"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}