A data feed allows organizations to deliver data streams from a server to a client automatically or on demand, the format of which allows for structured, current and updated information. The client typically uses this data for a variety of purposes, such as news distribution, media monitoring, cybersecurity, displaying products, or financial market information.
Structure of Data Feeds
Data feeds are often organized through JSON, Excel, XML or CSV files. This includes a repository, also known as a big index of data, that another party can consume. Data feeds often need very particular structures so that they can be easily consumed by a software. Each item in a data feed needs pre-set attributes that distinguish it from other items. For example, a news monitoring platform might want to consume data according to different blogs, and online discussions sources according to pre-filtered attributes according to different authors or from a specific geographic region. It is important to update data feeds regularly and include information for all fields of the data feed to avoid misaligned information.
Applications of Data Feeds
While data feeds are a general mechanism to transmit data from one source to another, there are many specific applications of data feeds.
- News Distribution: News and media monitoring companies, market research companies and businesses in consumer industries need news feeds to monitor the news and their brand in real-time. These news feeds include web content from a huge repository of sources, including blogs, forums, news articles and radio stations, often in the format of JSON, Excel, XML or RSS, (“Really Simple Syndication”). Examples of popular news feeds are those from Google News, Reddit or BBC.
- Online Discussions: Social media companies and businesses in consumer industries need data feeds to monitor their brand, stay on top of the latest news in their industry in real-time and gain insights on consumer trends. This includes data from online discussions across different message boards, review sites and forums. Companies such as Sprinklr, Crimson Hexagon and Salesforce (Radian6) use these types of data feeds to monitor online discussions on the web.
- Financial Markets: Data feeds for financial markets, especially high-frequency trading, require real-time updates to financial market data. As a result, high-speed data feeds transmit more reliable data faster, without delays for use in real-time data analysis. Bloomberg’s B-PIPE data feed, Thomas Reuters’ Matching Binary Multicast Feed and EBS Brokertec’s Ultra are examples of high-speed feeds that transmit data about financial markets.
- Cybersecurity: Businesses and security intelligence organizations need to monitor criminal activity to detect and prevent terrorist attacks, security breaches, and copyright infringements. In addition to monitoring and providing data from the open web, cybersecurity feeds can also pull data from the Dark Web, forums and paste websites that criminals and terrorists are using anonymously. Example of Darkweb networks that can be used for cyber data feeds include: TOR, I2P, ZeroNet, IRC and OpenBazaar networks.
- Product Distribution: Product data feeds help companies load large amounts of information about their products to electronic displays such as websites. These data feeds are often updated on demand, such as when a new product line arrives, and include crucial information such as titles, keywords in descriptions, types, colors, etc. While in some cases publishers and suppliers may hail from the same organizations, third-party sellers such as Amazon or Google Shopping utilize data feeds from many companies to publish the products on external platforms.
Guy Mor, Co-Founder and CMO of Webhose.io
Guy is the Co-founder and CMO of Webhose.io, a software company that provides an innovative web content collection platform used by hundreds of data analytics, cybersecurity and web monitoring companies worldwide. Before co-founding Webhose.io, Guy had previously co-founded Buzzilla Ltd., which develops software that helps brands track, monitor, analyze and extract insight from online content. Guy filled several senior positions in Buzzilla between the years 2010-2016, and still sits on the company’s board of directors. Prior to that, Guy co-founded and acted as CEO of TBWA DIGITAL - a leading digital advertising agency, acquired by TBWA Worldwide in December 2009. Guy has over two decades of experience leading successful data analytics and digital marketing ventures, as both a founder and a senior executive in various business development and product management roles. Guy has a BFA from the Bezalel Arts and Design Academy, specializing in Digital Media
“What is a Data Feed?” DataFeedWatch, Data Feed Watch, n.d.
“High-Speed Data Feed,” Investopedia, Investopedia, n.d.
Johnson, Tara. “How to Optimize Your Amazon SEO & Product Content,” CPC Strategy Amazon Marketplace, CPC Strategy, 3 June 2016.