Author Archives: Eran Levy

Structuring the Dark Web!

Posted on January 24, 2018 by Eran Levy

We’ve recently launched an exciting new addition to our dark web data feed (as featured on Betanews, ProgrammableWeb, and elsewhere): now, in addition to industry-leading breadth of coverage of the TOR network, we’ll also be structuring the extracted data so that it fits into a similar JSON format as our open web data feeds. The

Continue reading

Posted in Dark Web | Comments Off on Structuring the Dark Web!

What is the Omgili Bot, and why is it Crawling Your Website?

Posted on December 28, 2017 by Eran Levy

Hi there. If you’re reading this, it’s probably because you’ve run into Omgilibot – perhaps in your web analytics or server logs (user agent: omgili/0.5 +https://omgili.com) – and turned to Google to decide whether this crawler is a benevolent creature that should be permitted to do as it will, or something more nefarious that deserves

Continue reading

Posted in API | Comments Off on What is the Omgili Bot, and why is it Crawling Your Website?

3 Predictions for Web Data in 2018

Posted on December 12, 2017 by Eran Levy

2017 was a turbulent year: With Donald Trump shaking up the American political system, cryptocurrencies causing riptides throughout financial markets, and advancements in artificial intelligence sparking both anticipation and anxiety in the scientific world, the passing year seems to have been dominated by a sense of uncertainty and a sea change waiting to happen at

Continue reading

Posted in Big Data | Comments Off on 3 Predictions for Web Data in 2018

Richer Media Analysis with Broadcast News Transcripts

Posted on November 23, 2017 by Eran Levy

After a few long nights and some very tired developers, we’re proud to introduce the Broadcast Data Feed: transcribed, structured, and machine-readable television and radio programming from 1091 US TV stations, 21 Canadian TV stations, 83 Spanish-US TV Stations and 356 Radio Stations. This new product line is available and ready for you to use,

Continue reading

Posted in API | Comments Off on Richer Media Analysis with Broadcast News Transcripts

Quick Guide to News APIs

Posted on October 10, 2017 by Eran Levy

Monitoring mass media has come a long way since the days of the press-cutting agency. The bulk of today’s news is published online, while modern technology lets us store, index and query massive amounts of textual data in milliseconds. Digitization presents clear advantages for consumers, who can now read or watch the news from the

Continue reading

Posted in API | Comments Off on Quick Guide to News APIs

How To Get The Data Your AI Application Needs

Posted on September 28, 2017 by Eran Levy

Do you remember the saying, “For people who do not know where they are going, any path will take them there”? How things change! Today, it’s “No problem, let AI and machine learning figure it out!”. Artificial intelligence has advanced by leaps and bounds in recent years. Now, AI has beaten the world champion at

Continue reading

Posted in Machine Learning | Comments Off on How To Get The Data Your AI Application Needs

There’s An Exciting New Way to Access Dark Web Data

Posted on September 21, 2017 by Eran Levy

For the past few months we’ve been rolling out a new feature that’s already garnering a lot of excitement. Now that it’s officially live and actually being used by our customers, it’s time to officially introduce it. So please put your hands together and give a warm welcome to the Dark Web Data Feed! Basically,

Continue reading

Posted in Dark Web | Comments Off on There’s An Exciting New Way to Access Dark Web Data

3 Ways to Use eCommerce Product Data for Market Research

Posted on September 14, 2017 by Eran Levy

The web is an invaluable source of data when it comes to competitive intelligence, research and creating a go-to-market strategy. A simple Google search will reveal incredible amounts of public information about your main competitors, and connect you to mounds of existing research, financial analysis and other information that could be pertinent to your next

Continue reading

Posted in Big Data | Comments Off on 3 Ways to Use eCommerce Product Data for Market Research

How to Extract Data from Websites: Scraping Tools, DIY or DaaS

Posted on September 7, 2017 by Eran Levy

This is part 2 of our guide to web data extraction. Read part 1 to learn about the questions to ask before you start, or download the complete Web Data Extraction Playbook (PDF). Now that you’ve covered both the business and technical requirements for your web data extraction project (and if you haven’t, check out

Continue reading

Posted in Big Data | Comments Off on How to Extract Data from Websites: Scraping Tools, DIY or DaaS

Web Data Extraction Guide: 11 Questions to Ask

Posted on August 31, 2017 by Eran Levy

The following is an excerpt from our new Web Data Extraction Playbook. We’ll be publishing the second part next week, or you can grab the full guide here. The internet has become an undeniable force in our lives over the past few decades, changing everything from the way we do our shopping to the way

Continue reading

Posted in Big Data | Comments Off on Web Data Extraction Guide: 11 Questions to Ask

What is DaaS, BDaaS, DBaaS? And Why Should You Care?

Posted on August 8, 2017 by Eran Levy

The proliferation of data services has created a wide range of confusing buzzwords and acronyms – but at its core DaaS is still a meaningful concept. We are living in the age of everything as a service (EaaS?). What started as the simple and fairly easy to understand concept of software as a service has,

Continue reading

Posted in Big Data | Comments Off on What is DaaS, BDaaS, DBaaS? And Why Should You Care?

Machine Learning Showdown: Python vs R

Posted on August 2, 2017 by Eran Levy

Let’s say you have an amazing idea for a machine learning app. It’s going to be brilliant. It’s going to revolutionize the world of finance, mobile advertising, or… some other world, but it’s definitely going to revolutionize something. And gosh darn it, it’s going to be the smartest, most learned app the world has ever

Continue reading

Posted in Machine Learning | Leave a comment

Crawling the Dark Web to Detect the Next Market

Posted on July 25, 2017 by Eran Levy

Over the past few days, the internet has been abuzz with talk of the recent blows dealt by law enforcement to two major dark web “marketplaces”, AlphaBay and Hansa market; and the subsequent suicide of Alexander Cazes – the Canadian programmer-turned-criminal mastermind behind AlphaBay, who ended his own life in a Thai prison while awaiting

Continue reading

Posted in Dark Web | Comments Off on Crawling the Dark Web to Detect the Next Market