"Targeting & measurement as we know it is going away."- Angelina Eng
Angelina Eng, VP of Measurement and Attribution at the IAB, speaks with Allyson and Brett about advertisers’ overconfidence in data deprecation preparedness, agency staffing reconfigurations, and the need for stronger measurement foundations.
Together, they dig deep into analytics and attribution, and discuss why taking a holistic view of media and aligning on metrics is so important. From privacy and compliance, to the boom of retail media networks and the need to go beyond A/B testing, they discuss the key trends disrupting advertisers and marketers in 2022.
Allyson Dietz: Joining us today is Angelina Eng. Vice president of measurement and attribution at the IAB. If you don't know Angelina, she's been an active member of the digital marketing industry for over 25 years. Over the course of her career, she has supported over a 100 marketers define, build, and manage their digital media and marketing plans, and can often be found speaking at events, establishing standards for ad tracking and deliverables, and sharing best practices for emerging, media platforms. Angelina, welcome to the No Hype podcast.
Angelina Eng: Thank you so much. Very excited to be here.
Brett House: Welcome Angelina. You've had a pretty extensive career in the media industry. You've spent a lot of time at big agencies like Publicis and Dentsu, particularly in ad operations. How did that experience help you prepare you for this role at the IAB?
AE: Well, it was quite interesting. I think one of the main things that I was responsible was for data quality and ensuring that whatever our ad ops team was doing in terms of activating campaigns on behalf of our clients, was then making sure that we were collecting the data and transforming it. So normalizing it and so forth. So as part of that responsibility, I participated in a lot of IAB councils and committees around measurement standards, such as viewability, as we know it today and ad verification.
AE: And so it just kind of walked its way through in terms of making sure that I was looking at it from the analytics team's end. So once I got here, I took a lot of the best practices that I had on the agency side from both my media planning and buying experience, as well as my ad ops to focus on kind of the data pipes and the data signals and figuring out what's working, what's not working and what are some solutions that we should be looking at.
AD: And I saw, when you spent some time in the agency world, you spent some time on the Kraft business. And I also had the opportunity to work with Kraft a bit here in Chicago. And so I was kind of curious. I personally believe they're ahead of others as it relates to CPG with their kraftrecipes.com and what they've been trying to accomplish there. I just thought maybe you could speak to your experience working on their business and share a little bit about that first party data strategy?
AE: That was a great account to work on. I love the Kraft Foods business. One, it exposed me to the inner thinkings of how brands were thinking about their products in a non-advertising way. So our focus was primarily on recipes, supporting Food and Family Magazine at that time. It was one of the largest subscriptions in terms of food magazines compared to all the other print publications and websites.
AE: They had over at that time over 12 million unique users in their database, which was probably seven times bigger than what Food Network had at the time or Allrecipes. So it was interesting in terms of building out their first party data strategy. It was driving people see that Food and Family magazine, which we considered a high quality recipe magazine for free.
AD: Yeah, it's crazy. I thought that it was just such an innovative way to develop your first party data and really ahead of the time. I think now... At the time it there was no concerns about cookies and whatnot. And now with all the data deprecation, it's a top of mind concern, particularly for the CPG space. So I thought it was a really interesting example of how brands were achieving that goal, well before it was even a focus.
AE: Yeah. I mean, not only were they focusing on their individual products and brands, but I think this overall branding of Kraft Foods as an entity of high quality family recipes and ensuring that the types of recipes that they were publishing were focused around doing it as a joint family effort. So bringing in the children into the activity.
Kraft might be one of the early players to be sort of brand as publisher, right? A good way, not only to gather first party data to better capture information about your buyers, especially as a CPG company. I think of Red Bull as another player in that space where you almost forget sometimes that Red Bull started as an energy drink. Because they support a lot of activities that require a lot of energy, but you almost forget in some cases that they're a CPG brand versus a media company. So certainly interesting with Kraft. So the IAB just published the State of Data 2022 report. Super interesting. I read these every year. This year sort of focuses on the disruption to marketing measurement.
BH: And what I had thought was really interesting reading this was this focus on, almost a hubris coming from advertisers, for lack of a better way of saying it. In the sense of an overconfidence, in terms of how well prepared they really are for data deprecation and some of the massive disruptions happening in the data oriented marketing space. With 77% claiming they're prepared for cookies and identifiers and related things going away. But it seems like some of the insights from the report suggest that that confidence is in fact overconfidence. Can you talk a little bit about some of the key takeaways and if what I've just said was an accurate read of the 2022 report.
AE: Yeah, absolutely. All correct. And just to put things in perspective. There's a 15 point increase in confidence compared to the year before, across the board. We did both a qualitative and quantitative study. So not only we had a really extensive survey that went out to the community, but we also interviewed executives from over 20 companies across the board. Ad, tech, publishers, ad agencies and marketers. And I think one of the main things was this reliance on their ad tech partners. Many of them said that they believe that they are going to own the solution. So they're looking to them as the problem solver. And I think another point is around this wait and see mentality. With all the changes and all the news around privacy regulation and FLoC going away and having Topics API, there's a bit of ambiguity in terms of what exactly those changes are.
AE: And if in fact it's going to happen. And then the third reason I think that a lot of them are overconfident is they're just not ready to make a decision, right? So they have other things to focus on given what's happening right now, with COVID over the past two years, a lot of the social causes that have occurred. So their focus hasn't been on, what are these changes and how to go about it? Again, they're looking to their third party ad solutions. And some of those brands are looking towards their agencies for these solutions as well.
BH: Yeah, it's interesting because the three things that really jumped out from what you just said was, one, they're outsourcing it through third party. Which sounds like a diffusion of responsibility. Two, it's a wait and see approach. Right? Which can be a challenge considering how disruptive some of these changes from a privacy regulation and data deprecation might be to their future of audience reach and the ability to measure and analyze what's going on across their marketing spend.
BH: And then the third being, I guess that's an external factor in terms of the state of the world and them adapting to the realities of everything from war to the pandemic. That one's probably the most excusable. The first two, seems to me... Do you see that as a big challenge? And do you see that... What do you think that they should be doing to properly prepare themselves?
AE: Well, the first thing is don't take the word of the ad tech solutions. Ad tech partners that they've already have to solve. I think there are a few companies out there that are doing it really well and have a really interesting stack that they've put together, including you guys. Right? So from a Neustar standpoint, I think looking at it from beginning to end. Consumer management, consumer consent, first party data, measurement and attribution, looking at AI models, data clean rooms.
AE: Many of the companies out there that are actually serving ads, right? So think of the DSPs, the ad servers. The ad verification companies, those companies all rely on third party cookies. And what I'm hearing is, "Oh, they have their own first party solutions. So they're okay." Or, "They don't use third party cookies at all."
AE: That is a bit scary because right now our solutions do rely on third party cookies. And even if you don't use third party cookies, the tech that you're working with uses third party cookies. What we know today as ad targeting is going away. It's going to be more anonymous. We're going to rely on more first party data. I think that this is an opportunity for brands and publishers to work together around their first party data strategies.
AD: And Angelina, you mentioned, hings are going away as we know it today. And I think one of the things that really stood out to us is the report really points out that measurement as marketers know it today is going away and the industry is not moving fast enough to prepare. That's something that we've been talking a little bit about here just now, but I think I just want to take a step back and just talk about defining terms.
AD: There's lots of ambiguous terms in the industry or terms that mean different things to different people. And I think measurement happens to be one of them as well. And you're an expert in this space. I think measurement is oftentimes the word that we use to describe things like reach and frequency and television measurement. How many people or households are you actually touching when you execute a given campaign? But as you know measurement can go well beyond that. I'd love for you to talk a little bit about what measurement means to you.
AE: Well, I think in terms of the way that companies are measuring today is really a deterministic model, right? Where it's tied to a device, it's tied to a browser or it's tied to an individual or a set top box. So it's very device driven and some are using actual first party data. The way that we're looking at measurement right now is taking those signals and taking proxy signals to determine the attribution model.
AE: So figuring out if a user has been exposed, what types of engagement they have with the ad. And then how do you tie that back on the back end. That is more challenging now. We know that through Apple OS, you hear Facebook has struggled. A couple of other platforms have struggled. We see that opt-in rates are at, according to, I think it's Flurry, at 18%, right? That's a bit scary. Right? That's scary. So we're going to need to move towards a more inference, probabilistic modeling, overlaying a lot of information. And we're going to have to experiment. And I think that marketers and publishers are going to need to experiment because it's going to have such an impact, and a ripple effect across the board, from advertisers to ad tech, to publishers.
AD: And when you say ripple effect, what kind of effect do you anticipate?
AE: Well, I think that a lot of companies are going to try and focus on first party data. And there's only so much first party data that publishers and ad tech companies are going to be able to match. So the CPM rates are going to be really high for them. Which is a good thing, right? And hopefully it deters, minimizes fraud. But there is going to be a chunk of audiences that are anonymous, that you're not going to have any real significant individual data against. And so companies are going to have to struggle to figure out, what are the right approaches to creating those audiences and targeting them without having to waste a lot of dollars.
BH: Yeah. And that ties into things like contextual. And I know you have recommendations on how you can better target contextual advertising and actual segmentation scheme is built around that model. But it's an interesting topic because it's sort of... Allison and I were talking about this the other day, this sort of bifurcation of the open web versus the sort of closed web, right?
BH: These gated, walled garden, publisher properties, whether the social media giants, the Googles, the Amazons, the Hulus, the Facebooks, et cetera. And there's a concentration of power, and a concentration of audience amongst a handful of very, very powerful players in the space. Do you think that has a big part in what is going to be an increase in cost potentially, because there are fewer places to go to access broader wide audiences across the web?
AE: Absolutely. I think those that have really large first party data are going to be the ones that brands are going to lean towards first. Right? So kind of attack, plan of action is to find those publishers or even walled gardens. What's interesting is that, with the walled gardens is they can provide a bit of a closed loop measurement reporting framework, but the problem with that is many companies are going to have to work with several walled gardens and they're all not going to be connected.
AE: And so there's going to be still some fragmentation. We have that today, right? You can't really tie Twitter data to Facebook data, to Amazon data without an underlying third party cookie right now. I think the data clean rooms are going to be great for marketers who are... Where those companies are going to be able to formulate relationships with each of those walled gardens and figure out how to actually map the data together. And do it using AI machine learning for modeling. Probabilistic modeling is the key.
AD: When you talk about the data clean room use case and being able to match together different data sources, Twitter to Facebook, so on and so forth. Both you and I have this measurement background. I always think about the use case of being able to do that so that you can see things like reach and frequency across those platforms. So that you can stitch together the customer's journey and they're ultimately driving their conversion.
AD: Is that the key use case that you see in terms of the need for stitching together that data, or what are those use cases that you think about when you think about the clean room?
AE: I think about reach and frequency. I think that's a huge part. I think that's table stakes in terms of what clients want. They want to understand duplication or over duplication, [00:17:00] if you will. Or where the overlaps are. I see measurement in four buckets.
AE: There's audience measurement like reach and frequency. Then I see ad delivery itself. Impressions being served. Is it viewable? Is it not viewable? Was it viewed by human or non-human? Then you have ad engagement in this interaction. Is it attention metrics, which is also something that I'm hearing a lot about in the industry. And then outcomes. So that could be sales, conversions, leads, tying in store to online, different needs for different clients. But I think that each of those buckets will have its own challenges.
BH: Just to extrapolate on that topic. So you had the ad measurement, ad delivery, ad engagement, attention metrics, and then you had outcomes. Are we leaving out, and I guess this gets into this conversation around measurement versus analytics or measurement versus attribution, right? Because outcomes are in a sense defined by a deeper view of analytics and attribution that media plays a part in, but isn't the entire pie, right?
BH: There's existing customer propensity, there's seasonality, there's economic impacts, there's pandemic type impacts that are affecting people's purchase behaviors. In store, physical advertising that are driving people to impulse purchases or otherwise. Is the IAB thinking of analytics and attribution as it is tied to this sort of holistic view of understanding where media plays and where media's most effective?
AE: I think we look at it in lots of different views. There's a focus on specific channels. For example, right now we're focusing, we have a task force around intrinsic in game. So 3D environments where you have a billboard or a automobile that's skinned with a logo or a basketball court or a baseball field. Right? And then we have this idea... So we look at that. We have video measurement, and then we're looking at it holistically from a cross channel perspective of figuring out what are the ways that we can count data consistently across those different channels and making sure that we're aligned on definitions as well as methodology. And then I think diving into what are the proxy signals that we're getting, and then figuring how, from an attribution standpoint, will we be able to stitch it?
AE: It's really unclear right now, especially given what little we know about privacy sandbox and conversion measurement. It's constantly changing. It's evolving. Not a lot of people are leaned in. It's predominantly, mostly tech companies. And then you have all these other various platforms, walled gardens that are trying to do their own thing and coming up with their own solutions. But our approach is to figure out what are the commonalities and where we can actually align... We use these terms, common currency or common transaction KPIs. And I think that's one of the things that we have to focus on first is, let's come up with a set of standard metrics that we can all agree on and then expand from there.
AD: You're talking about standard metrics, obviously. I think that's really interesting. Alignment around key KPIs that everyone sort of agrees to. Not only what are they, but how do we get there? And I think that's really interesting, but one of the things that I found really interesting in the report itself was that 60% of industry leaders really expect ad campaign measurement to be affected, but they're not really taking any action.
AD: And we talked a little bit about this before, but I'd love to hear as it relates to measurement, why aren't they taking action and what should brands be doing? In particular, what should brands be doing as it relates to measurement and how they should prepare for the changes in the ecosystem and how they impact measurement?
AE: Well, I think they should look at standardizing the measurement approach across all channels. That's one. I'm still hearing that agencies or their counterparts, whoever's in charge of activation is still pretty fragmented. You have the social team doing their own thing compared to the search team, programmatics running their own campaign. Some of those agencies are bringing all that data together into a data clean room, but not all of them are.
AD: Is that because that's how they buy media today? I mean, is that what's driving it, is that, you said a social team, you've got a TV team. These are different teams.
BH: Yeah, these teams have budgets and they want to defend their budgets, right?
AE: Yeah. Well, we can have a whole conversation of what are some of the challenges on the agency side, but it really comes to resources and talent, right? And process. I think the agencies have a responsibility to reconfigure their staff so that there is an integrated, centralized reporting approach. Which again, doesn't happen for all clients. And I think that clients need to pay for those resources.
AD: Spoken like a true agency person.
AE: But I mean, it's true. I mean, I can't tell you the number of times where we would have a media team of 60 people, and this is across different channels. And they would only support two analytics personnel. And so think about that. How can two people aggregate all that data, normalize all that data, process all that data and come up with a cross channel, omnichannel measurement analytics approach. It's impossible.So it does come down to staffing and resources and making sure that you have the proper team in place.
AD: What else do you think marketers should be doing to prepare? Alignment obviously is the first step, but what additional things do you think are important?
AE: Don't take your partner's word for it, that they have a solver, that they're working on it. Really, truly work with them, map out your data, understand your platforms, figure out what your stack looks like. How do you bring it all together? Is it a data clean room or are there specific things that you can do as simple as standard taxonomies around your naming conventions. Campaign names, placement, creative, some of those things that I think that a lot of companies don't really take advantage of some of the tools that they already have available to them.
AE: And many of them don't know what to ask. So I think, don't just rely on your ad tech solution. Don't just rely on your agency. It really needs to be a collaborative discussion. You have to audit the entire process. How's that data being collected? How is it being normalized? What are the gaps in there, right? Are you using too many site serve placements? Too many, one by one pixels. I mean, I'm getting a little too tactical here, but truly if you want to see improvements in measurement, you have to have a really good foundation. Many of them don't.
BH: So there's a lot of hype in the marketplace around... As we've been talking about data deprecation's impact on analytics, attribution, measurement, we're going to put all three of those into slightly different camps. And we'll dive a little bit deeper into sort of the attribution topic, because I think what we've been talking right now is media measurement.
BH: But if media is only 15% or 20% of the influence, in terms of a final purchase by a consumer, are you looking at the other 80%? And do you understand what that is? Whether again, from a CPG perspective, it could be an in-store type of, either placement on a shelf, endcaps, special offers, et cetera.
BH: To the out of home, to the kind of the entire rest of the person's experience outside of just simple media exposure. Because I think there's just a tendency to, especially from the agency side to over-credit the impact of media on that purchase behavior. And so everybody's claiming they're driving ROI and ROAS and all these other things, but at the end of the day, it might only be 15% of what's really influencing that fundamental purchase by the consumer. Right? I mean, do you think that's an accurate statement generally?
AE: I think so. I think currently there are ways to understand what possibly... Or at least I understand how to create some digital AB testing, creative segmentation analysis to see those lifts.. I do think it's harder to bring the rest of the marketing ecosystem into the formula, especially given the fact that we can't even get digital to a really good place.
BH: We can't even get the digital media part of it right.
AE: Correct.
BH: Yeah.
AE: We recognize it at the IAB. Right? We recognize that there are definitely other advertising marketing channels that are being used by marketers today. I think for us is really focusing on as things are moving towards digital or if we can get more insights coming directly from the audience and the consumer around what they're purchasing through surveys, through panels,through focus groups and overlay that, that would be great.
BH: Yeah. Because then you have a look into propensity. Are they more likely to buy this? Yea or nay? And are they going to buy it without being advertised to in the first place?
AE: Yeah.
BH: Right?
AE: Absolutely.
BH: You kind of hit on this, is that there's certainly what I like to call is a “back to the future” moment right now. Where some of these very, we'll call them old or well established legacy analytics approaches, like marketing mix modeling, classic example. It's been around for, what Allison? 40 or 50 years?
AD: Probably longer.
AE: like 60.
BH: Yeah. 50 or 60 years. And it seems like a lot of brands are defaulting to solutions like that and sort of throwing their hands up when it comes to more advanced solutions that are going to be more accurate in predicting purchase behavior. Like multi-touch attribution. What do you think of that and some of the trends that you've seen recently?
AE: Well, I think if COVID and this war has any indication, clients have to shift pretty quickly or evaluate their marketing mix frequently.
AD: Speed matters.
AE: Speed matters, timing matters, seasonality matters. I think social and political influences have had a significant impact on the way that media and product are being sold and consumed. And so marketers have to be more reactive.
AD: So you mentioned this a second ago. Brett mentioned the “back to the future” concept. And in addition to marketing mix modeling, a lot of marketers are leaning on other tools like AB testing. Almost as a replacement for attribution or MTA. What role do you think test and learn plays in the overall mix? Is there a reason why you might use one versus the other or do you see a role for all three in a marketing toolkit?
AE: I think each advertiser needs to test and learn all the different options. I'd love to see clients go beyond AB testing. Going back to my Kraft Foods day, we had a fantastic analytics team where we focused on fractional factorial methodologies. And it allowed us to create various different ad units and messaging and pinpoint exactly what are the elements that made a successful ad unit.
AE: And then we applied that to audiences as well. And every time we conducted this test, we saw a 25% increase in subscription rates for Food and Family Magazine. And we did this every quarter. Because we understood that seasonality had something to do with it or perhaps the types of recipes that we... Or the images that we were promoting. Was there an image of children in our magazine versus just the recipes themselves? So fractional factorial methodologies, AB testing, segmentation, creative. I think all of those are on the table. I do think it will be harder though when the cookie goes away, right? You can do AB testing on your own site, experiment with your own audiences and then take those learnings and apply them out in the marketplace and see what you get back in return. And I think.
AD: You still need that holistic approach, right? To be able to say, if you're doing it in a little mini test environment, AB testing in a test environment, you do need to be able to kind of have a more comprehensive view as to whether or not it's actually driving performance or not. So, yeah, it's interesting. Because I think what we hear a lot about in the industry is, "Oh, attribution is dead. MTA is dead." And I'm curious, what you think about that. Is that the case? Do you think solutions like MTA are no longer relevant in today's world with cookie deprecation?
AE: I think, if you talk to clients, it has to exist. Otherwise they're going to have such difficulty in optimizing and figuring out where they're going to spend their dollars and who they're going to spend their dollars with. And how much to spend. I don't think it's completely dead. I do want to raise the alarm bells that it's just going to be more difficult.
AE: I think that, again, if you're working within a closed ecosystem like a walled garden or a retail media network, they'll be able to give you closed loop reporting within their own view. Right? The question is, can you stitch it all together holistically? I think the holistic is what's going to be hard.
BH: Yeah. Because the methodologies between those retail media networks, walled gardens, different publishers are going to be different. They're not going to all just... Speaking to this universal application of sort of analytics and measurement. You're not going to get them all agree to measure and grade their own homework in the same way.
AE: Or get the same data from
BH: Or get the same data. Yeah. And I guess it ties back to, so cookie deprecation,MAID deprecation, other forms of data deprecation for privacy purposes. What about, and I think this hearkens to this, we talked about earlier about this notion of the clean room environment. Where you can still leverage a lot of this rich data without actually revealing or moving personally identifiable or otherwise information back and forth, right?
BH: Where it lives in this completely vaulted, protected, limited access environment that really doesn't move data from point A to point B.Add that with the machine learning capabilities that are obviously becoming more advanced daily. Do we see a future where you're still going to be able to have access to rich information around consumers? It just won't be breachable, right? So something beyond the cookie that enables you to still have the accuracy from a targeting perspective, from a measurement perspective, et cetera. But is free of those concerns. .
AE: Absolutely. I think that the thing though is making sure that the quality of data is there, right? So you hear the saying, garbage in garbage out, right? We need to make sure that the AI models or the algorithms have very minimal bias in it. And so what data you're collecting, how that's being processed, do you have enough of the cross references in terms of the audience segment? Right? A really good example. One example that we had, when we published the AI standards, best practices for bias and AI for marketing, a few months ago, was you can have a really nice sample size of women, right? And then you can have a nice sample size of Asian, but your cross reference of Asian women may not be enough. And so as the end user, you might not be aware that the sample size is too small and that could throw off the model.
AE: So it's up to the companies that are pulling this data together, to think about all the different permutations of data that they need in order to be more accurate. So I think that's one caveat is, I think the power of AI and machine learning is going to help a lot of companies. Think about how AI is being used in the medical field right now. It's freaking phenomenal. Right? And so I see more companies leaning towards that. Problem with that is that a lot of the big companies are going to be able to leverage that, but not a lot of the small to medium sized companies out there.
What three trends do you believe will disrupt the industry in '22 and beyond?
AE: Well, privacy and compliance, huge, huge factor. I think more companies need to lean in and make sure that they’re CPRA prepared as well as monitoring what's happening in Colorado, in Utah and Virginia and whatever, DC. Whatever other state. Because there's going to be requirements that are not consistent from each state that you're going to have to comply with.
AE: And that's going to have to spill over to your ad tech partner and how they process each state on their own. Secondly, I want to go back to that retail media network. I think retail media, the growth and the boom of retail media networks is super fascinating. Walmart started off many years ago and now you have Randal, you have CVS, you have Best Buy, Peapod, Kroger. It's crazy how many retailers have decided to monetize their own first-party data. I think what they're facing is some of the challenges that publishers had faced many years ago? And so we have this, we had a track over at ALM called, are they publishers now?
AD: Angelina, I'm curious if you have any final words of advice in terms of the future of measurement or the future of advertising. Things that you would, if you were in their shoes, what would you be doing to prepare for this future.
AE: Well, I definitely think that everyone's voice should be heard in terms of what they think is going to be the impact of some of these proposal changes that are coming down the pipe. Including topics and FLEDGE and so forth. So that would be participate in the W3C, participate in the IAB’s browser OS task force. Where a lot of these conversations we're having.
AE: Where we're providing a platform for all companies, brands, marketers, agencies, publishers, to hear what those changes are. And then we have dialogue about what the impact is and figure out what are ways that we're going to test these things out. But I think their voice has to be heard. And we saw that would FLoC, right? So FLoC no longer exists because the marketplace raised their hand and said, "Hey, this isn't going to work. Here's some proof. This is our feedback. Both from a technical standpoint, as well as from a business standpoint."
AD: Yeah. So do the work. You mentioned that earlier. Do the homework. Really understand how this impacts you and then take a stand and fight for what you need. I think that that makes a lot of sense.
BH: And make some of these players to do overs, right? Like with topics. From FLoC to Topics. Now we're going to all vote on Topics to see if that's going to be a standard going forward. Right?
AD: Time will tell.
BH: Time will tell.
AE: What are your thoughts about Topics?
BH: I put it in the FLoC bucket, honestly. In terms of... Well it's... No, it's not dead on arrival and it's not dead yet, but to be determined, right? In terms of will this really work. And I think that feedback mechanism is at least starting right now, but it's an alpha, it's not even a beta. It's an idea. Right? And you've got to be able to execute on that idea with all parties involved, agreeing that this is a good approach. So yeah. I think I've got a wait and see, speaking of wait and see approaches, on whether or not this is going to go the same trajectory as FLoC went.
AE: See, what I think is the challenge is that perhaps the Chrome team should spend several days with people from the industry, from the business side, not the engineers, not the developers, possibly product, right? Product teams or product leaders. But I think that spending time with publishers and advertisers and agencies around exactly how do they actually do business every day?
AE: What are the things that they're implementing? How are they looking at the data? How are they creating the audiences? What's working? What's not working? It's so hard to document all the business use cases. And when things like Topics come up, I can anticipate all the questions that are coming up. Like, "Only 350 topics?"
AD: Is that enough?
AE: Is that enough? Why is it three weeks? Why don't we only get one? Is the user going to only see one type of advertiser for the entire week? Right? So is it just going to be auto ads all week or what? So there's just a lot of questions about these proposals. And I think it's again, worth for the business leaders to lend their voice.
BH: Yeah. What's the scale of each of these topics. Do we get the audience reach? What's the accuracy? Obviously the privacy implications is kind of the fundamental solve here, but are we replacing third party cookies with something that is superior or grossly inferior, I think is the question we ask ourselves, right?
AD: Yeah. Is this another bandaid?
BH: Yeah. Well, awesome conversation.
AE: Thank you for having me, this was great.
AD: Yeah.
BH: 100%.
AD: Really enjoyed chatting with you. Thank you again, Angelina. It's always a joy to chat with you.
AE: Thank you.