Why (and How) to Monitor RSS Feeds in 2018

Posted on March 27, 2018 by Guy Mor

Rich Site Summary (RSS), as a web technology, has been around since the turn of the last century. But is it still relevant in 2018, is it going to stay around for much longer, and how can it still be useful in today’s online landscape? Our answers to these questions are yes, yes, and read...

Continue reading

Posted in API

Meet the Online News Archive: Time for Some Historical Perspective

Posted on March 12, 2018 by Guy Mor

Today we’re very excited to announce the latest milestone in our journey to make structured web data easily accessible to every organization, developer and researcher: the Online News Archive has now been officially launched!   TL;DR version: it’s a massive database of online news articles in structured format collected from thousands of sources in over...

Continue reading

Posted in News

How Artificial Intelligence Can Bridge the Gap between Technology and Hype

Posted on February 12, 2018 by Guy Mor

If you read business or tech publications, you’ve probably heard about the ‘explosion of data in the business world’. There has certainly been no lack of voices shouting about it from every rooftop: That a claim has become clichéd does not, however, make it inaccurate. It is true that the internet, digitization, storage and other...

Continue reading

Posted in Machine Learning

Structuring the Dark Web!

Posted on January 24, 2018 by eranl

We’ve recently launched an exciting new addition to our dark web data feed (as featured on Betanews, ProgrammableWeb, and elsewhere): now, in addition to industry-leading breadth of coverage of the TOR network, we’ll also be structuring the extracted data so that it fits into a similar JSON format as our open web data feeds. The...

Continue reading

Posted in Dark Web

Financial success using AI and Time Travel

Posted on January 18, 2018 by Ran Geva

Wait let me explain. I can explain every part of this click-bait title, it will make sense I promise. So, A great philosopher named Homer Simpsons once said: “Trying is the first step towards failure” And I agree, however Failure is the first step towards success. Learning from past mistakes is a crucial step to...

Continue reading

Posted in Machine Learning

What is the Omgili Bot, and why is it Crawling Your Website?

Posted on December 28, 2017 by eranl

Hi there. If you’re reading this, it’s probably because you’ve run into Omgilibot – perhaps in your web analytics or server logs (user agent: omgili/0.5 +https://omgili.com) – and turned to Google to decide whether this crawler is a benevolent creature that should be permitted to do as it will, or something more nefarious that deserves...

Continue reading

Posted in API

3 Predictions for Web Data in 2018

Posted on December 12, 2017 by eranl

2017 was a turbulent year: With Donald Trump shaking up the American political system, cryptocurrencies causing riptides throughout financial markets, and advancements in artificial intelligence sparking both anticipation and anxiety in the scientific world, the passing year seems to have been dominated by a sense of uncertainty and a sea change waiting to happen at...

Continue reading

Posted in Big Data