Tag: featured

Posted on

Big Data: What I am thankful this Thanksgiving!

big data at iscpif

Big Data platform at ISCPIF: This list represents what I am thankful to this Thanksgiving

  • Elasticsearch You know for search!

This is the heart and the soul of our discovery and curiosity. The first step of every data analytics is finding the right data. Behind every great data discovery there must be a great search engine 🙂 And this is where Elasticsearch comes to the rescue with all its advanced real-time data analytics and powerful full-text search abilities built over Apache Lucene.
ISCPIF Big Data platform uses Elasticsearch and its built-in distributed and high availability feature with over 900 million documents indexed in order to facilitate data discoveries and explorations for our researchers and scientific partners.

So I am thankful to Elasticsearch for the past 2 years, which made my life much easier and made me a better engineer.
Continue reading “Big Data: What I am thankful this Thanksgiving!”

Posted on

Paris before and during terrorist attacks

 

Night of terror

The data from Twitter shows some upsetting stats about Paris terrorist attacks on 13 November.

As it can be seen, the query for terrorist attacks has no result before 22h30. Unfortunately this shows even without any knowledge of the event itself (media or news), it is indeed possible to assume something must have happened related to the requested queries.

Paris after attacks

Continue reading “Paris before and during terrorist attacks”

Posted on

Building a real-time news analytics

It is always interesting to find out what exactly is happening around the world as they are happening. This feature of knowing something as it happens is so called “realtime” or “near realtime” because of the network latency, delay for processing the data, delay for visualizing and streaming the data, etc.

Now imagine you can monitor most international news agencies on Twitter in real time. What I have been developing for the last few months is an application that allows you to not only get important news and highlights around the world as they are happening but also persists the data for the window of 24 hours so you can always have the ability to read important news and highlights that have already happened.

Continue reading “Building a real-time news analytics”