When Wading into the Bidstream, Beware the Currents

Last Updated: December 16, 2021

Programmatic media buying has given rise to the bidstream, a dataset that could prove incredibly valuable for marketers looking for deeper insight into online audiences. While the bidstream brings opportunity, marketers must also be aware of its technical hurdles and ethical limitations, shares,  Erik Matlick, co-founder and CEO, Bombora and Ameet Shah, VP, Global Tech and Data Strategy, Prohaska Consulting.

With the rise of Real-Time-Bidding (RTB), every impression opportunity contains a huge set of data points about valuable user behaviors and characteristics. The complete reporting output can provide similarly fertile ground for advanced analytics and audience insights. This broader data set generated by RTB auctions — won, lost, or passed on entirely, across vast swathes of biddable online audiences — has come to be known as the Bidstream, and it represents a significant opportunity for all kinds of marketers and data aggregators looking to understand audiences. 

While the opportunity is great, a few technical hurdles and ethical boundaries prevent many interested parties from simply tapping into this data. The future of bidstream data, and the risk and reward in using it, comes down to understanding which platforms have legitimate access to the data, and how increased regulation affects the use of that information.

Also Read: Will CTV Advertising Take Over Mobile in 2019?Opens a new window

What’s in the bidstream?

Log-level data (the most granular record of every data point evaluated in an RTB auction), is generated by the various tech vendors (ex. DSPs, ad servers, and SSPs) involved in the transaction. Key data points include the location (IP, latitude/longitude, zip), device (OS, connection type, carrier), site (full URL, app name, ad slot position, above/below the fold,), as well as demographics, anonymized when available (age/DOB, gender). These data prove valuable to marketers and their technology partners at every stage, as they can drive insights that lead to smarter and more efficient customer awareness and acquisition.

Data is accessed across the ecosystem

  • Advertisers buying through a DSP can gain log-level reporting data for impressions they have won in auction, though many platforms will apply an additional cost to set up and maintain a reporting feed.
  • Marketers and agencies housing their own bidders and actively participating in auctions gain access to their impressions, as well as to the auction logs where they bid and lose, or where they simply observe without bidding at all. The broader, more transparent data generated by the bidders directly is a major selling point of the bidder-as-a-service model that has grown in market share in recent years.
  • Platforms (DSPs and SSPs) that house and administer auctions create an exponentially greater reporting output spanning the entire network of buyers and sellers. Platforms can’t resell the data, or use it for derivative commercialization.  They can only apply the network-wide information to power their system intelligence, including buying algorithms and machine-learning reporting capabilities.

Identity Management is a Growing Business

With the majority of internet usage moving from desktop to mobile, tablet, IP-based television and beyond, Identity Graphs (large tables mapping anonymous households and individuals to their connected devices) have emerged as a synching solution for marketers looking to track and message high-value users across disparate environments. 

DSPs, DMPs, and data companies can use signals, like device ID in the bidstream, to build up their own identity graphs, though tech giants like Google, Facebook and Amazon have a distinct advantage here simply because the vast majority of people are logged into their apps and services across laptops, phones and television-based streaming products. To counteract the threat posed by these large and increasingly walled-off competitorsOpens a new window , independent companies are teaming up to link up their data assets and establish competitive, scaled identity-mapping solutions Opens a new window for marketers. This is especially relevant to B2B buyers, in that the research enables a connected profile of the buyers as individual people.

E-commerce and Offline Attribution

While advertisers can use bidstream-based reporting to track their customer’s path-to-purchase on their owned and operated e-commerce sites, data-sharing agreements with online portals like Amazon and Walmart open up a valuable new world of attribution for big retailers.  By synching the anonymous user-level data with purchase history on e-commerce platforms, advertisers can empirically prove that their ad spend is driving online sales across their key digital channels. 

Similarly, because offline purchases can be tracked to user-reward records (phone number, email, home address), there’s a growing ability to follow shoppers across the ecosystem (on- & offline), message them at various points, and confirm the relationship between ads and purchases. 

Also Read: Line It Up: Use Display and Native AD Targeting Tactics to Hit Your Business GoalsOpens a new window

Consumer Privacy and the Next Wave of Data Collection

While an individual’s identity is kept carefully concealed well before any associated data makes its way into the bidstream, an Identity Graph, or an attribution exercise, there is a growing level of concern about how much personal information is made available to characterize consumer behaviors and drive ad decisioning. It’s a priority for lawmakers and tech companies to draw the right line, including which information belongs to the individual and what limits are in place around data collection, especially as more and more of our devices and day-to-day interactions become connected and trackable. 

Data and Privacy Regulation

While much has been written about GDPR and its impact on marketers and consumers, the primary takeaway is that individuals are now in control of their personally identifiable information (including location, devices, and anything else that can be tied back to a user).  Publishers must obtain consent (usually from an opt-in form) before collecting any information on a site visitor, and a record must be kept of how data is collected and stored, along with a disclosure of any third-party companies that data is being shared with. 

GDPR has provided guidelines for regulations that are currently in nascent stages in the U.S.  In particular, the California Consumer Privacy Act (signed into law in 2018, going into effect in January 2020), reinforces consumers’ right to be told about any personal information to which sites have access, with the ability to access and delete any personal data being collected and/or sold. One primary difference here is that CCPA puts the onus on users to opt out of data collection, whereas GDPR requires opt-in consent before any data collection can occur.  At least nine other states are also pursuing legislation, as well as the federal government. Ideally, this should be regulated federally as opposed to individually by state, because the latter scenario would be very challenging to manage.

To complement legislative rollouts, the digital marketing industry maintains guidelines around data usage, and trade groups have recently teamed up to push for government regulations to address the matter. The latter initiative is good news for consumers, though industry insiders are also incentivized to influence the laws to avoid some of the more stringent (and privacy favoring) GDPR-style rules that can impact ad revenue. 

The Future of Data Privacy: CMPs, User Intent, and Privacy Controls

One pressing topic fueling data privacy conversations deals with the definition of user consent. GDPR law requires businesses to clearly declare their intent to process data in order for a user’s opt-in consent to be deemed legitimate, and there is confusion about whether a simple “Agree/Deny” prompt on a publisher site complies with this requirement. Further, publishers must allow users to manage their own privacy settings with the option to opt out of data collection at a later date if they so choose. CMPs (Consent Management Platforms — secure technical layers that help sites collect, store and manage user data preferences) have gained traction as catchall compliance solutions for publishers, though it’s not clear how obtaining user consent on one site translates to the large network of partners downstream that a publisher shares data with in order to do business.         

Bidstream data came up several times at the recent LUMA Partners Digital Media Summit, a gathering of CEOs from across the MarTech industry. One solution provider’s CEO minimized the value of bidstream data while speaking on stage, based on the veracity of the signals. Meanwhile, another executive from a major agency was quick to point out that issues of compliance and consent will deter marketers from using the data.  

Also Read: Why Fintech is the Next Ad Industry Disruptor: Creating Offline Transaction-Based AdvertisingOpens a new window

Closing Thoughts on Data Collection and Monetization

While the bidstream’s ever-expanding wealth of data presents potential value, marketers must address pressing ethical and moral questions about consumer privacy. Because these issues of surveillance and privacy affect all of us, it is vital that we (marketers, agencies, tech, publishers, and trade groups) establish firm rules around data collection in a privacy-first manner so that trust can exist between marketers and consumers.

Co-author: Ameet Shah VP: Global Tech, Publisher, & Data Strategy, Prohaska ConsultingOpens a new window

comp_ameet_shah_vpglobal_tech_publisher_data_strategy_prohaska_consulting_5d95db308665c image

​​​​​Ameet Shah works with Prohaska Consulting clients to help them more effectively leverage their data assets, optimize their ad technology stack to create additional capabilities, and implement yield management solutions.  He draws on more than 25 years of experience with 15+ years in digital media.

Erik Matlick
Erik Matlick

Founder and CEO, Bombora

Erik Matlick guides vision and corporate strategy at Bombora, bringing over 15 years in founding, board and executive management experience. An online performance marketing pioneer, Erik’s insights about the confluence of data analytics, media operations, ad serving technology and sales and marketing processes are the driving force behind Bombora’s unique value proposition.
Take me to Community
Do you still have questions? Head over to the Spiceworks Community to find answers.