What iOS 27 Means for Meta Ads Before Black Friday

[00:00:00] Tony: All right. Hello, hello. What's up, Steve? How are you?

[00:00:05] Steve: I'm great, Tony.

[00:00:06] Tony: Oh, you're doing okay? Well, uh, that's good. Glad news. I'm, I'm glad to hear that you're doing okay 'cause we have a, um, a potentially, uh, interesting, um, tracking curiosity, uh, that's, uh, bubbling up in our world. And, uh, and the topic of the day is iOS 27 and the new release, uh, the new A- Apple operating system release.

[00:00:27] And What, if anything, we, uh, need to be mindful of for our, uh, for our advertisers. So that's what we're here for. So I'm, I'm excited. Are you ready?

[00:00:39] Steve: I'm excited about this too. I think it's an important thing now that it's crept up and we're coming up on Black Friday and Cyber Monday. This is something to really kinda dial in, check your accounts.

[00:00:48] Tony: Yeah.

[00:00:49] Steve: Um, and make sure things are operating correctly-

[00:00:51] Tony: For sure ...

[00:00:51] Steve: going into an important point in time of year.

[00:00:53] Tony: Yeah. Yeah. So I think like the shape of the conversation that I'm hoping to have today is kind of like what's happening, um, when, uh, did it happen, kinda like the timeline, and then what's the impact and what, uh, what, what, uh, what do we focus on?

[00:01:07] What do we think our, our advertisers and our, our audience should be paying attention to? So, um, so let's start with, uh, kind of like the, the big picture. Um, let's talk about what, uh, what, what changed. Um, what's-- what is happening with iOS 27 specifically? Mm-hmm. Uh, and, uh, help me, help me understand it in sim-simple terms.

[00:01:29] Steve: Right. So Apple obviously updates its operating systems. Uh, we saw this way back in 2021 with iOS 14.5. Oh, we don't

[00:01:38] Tony: speak about that here.

[00:01:40] Steve: Rougher time. But when Apple does this, Apple is cool in that they like privacy, but sometimes that can be overly, uh, private and to the point where you lose signal.

[00:01:53] You're no longer able to see if a purchase conversion happened or tie it back to a particular ad.

[00:01:59] Tony: Sure.

[00:01:59] Steve: So when you lose that signal, you, first of all, might have worse looking performance on Meta, um, in that you no longer have these conversions that actually happened- Sure ... and Meta's not seeing them. And then you have a worse feedback loop.

[00:02:15] So Meta is not able to see and determine that a purchase happened, so they lose the ability to determine who's a good customer to potentially show your ad to.

[00:02:26] Tony: Right.

[00:02:26] Steve: Or their ability to do so is decreased since they're missing some of that signal.

[00:02:30] Tony: Right. Right. So sort of two, kind of two impact vectors, if you will.

[00:02:35] One being kind of let's call it attribution- Yes ... so the, the platform's reported impact.

[00:02:40] Steve: Right.

[00:02:40] Tony: And then the other being targeting.

[00:02:42] Steve: Yes. Great.

[00:02:44] Tony: Yeah. So signal impacts, uh, impacting those two things. I think for me, when I think about it, I, I'm always trying to wrestle with the idea of, um, do we actually have like, do we actually have fewer people buying, uh, AKA is there, is there an attribution problem?

[00:02:58] Um, or, uh, do we have like a targeting issue? Are we actually reaching, uh, less people or reaching people less efficiently? So it's interesting to separate those two things. So let's, let's talk more specifically. So I, I'm-- I guess maybe I have been, uh- In the weeds here for, uh, last whatever, um, trying to be able to articulate this, this change.

[00:03:19] Like what is actually happening mechanically, um, with iOS, uh, 27? And, and also how does that compare to, to the, the, um, unspeakable, um, iOS, iOS 14? So let me hit you with this and you tell me where my understanding is right or wrong or indifferent. So Um, iOS 27 is the, the particular change, uh, relates to a technology called WebKit, uh, which is a browser, like a browser rendering engine that's specific to Safari, also renders web pages, um, like inside apps.

[00:03:50] So it's, it's really browser-based. Um, and there is, uh, an, an ecosystem of, um, companies in the world, uh, called, um, um, identity graph providers, or essentially companies that, uh, sit in the, the background of the internet, and when you log into a website, uh, or fill out a form, your email address, as, as one example, is getting captured in this, in this database, and these identity graph providers, um, use that to sort of connect the dots in the advertising ecosystem between, um, a per- uh, a person and an ad interaction.

[00:04:27] So as I understand it, the iOS 27, um, uh, focus was focused in on WebKit and focused in on, um, detaching these identity graph providers' ability to capture that information. Um, help me say if that's right, wrong, and different, or if you would describe it in any different way.

[00:04:47] Steve: That, that's basically right. Yes.

[00:04:49] Absolutely.

[00:04:49] Tony: Wow.

[00:04:52] Steve: You read up on it quite a bit, I think, since we've had this issue pop up.

[00:04:56] Tony: Ba- basically right. I think that's the best compliment you can get from Steve.

[00:05:00] Steve: Yes. Um, I, I didn't dig into as many details as you, honestly-

[00:05:04] Tony: So- ...

[00:05:04] Steve: um, with respect to this.

[00:05:05] Tony: Okay, cool. So, so I think I have, like, the rough shape of it.

[00:05:08] Um, I wanna try to, like relate it to iOS 14, 'cause they, they are, like both privacy related, but sort of mechanically different. Um, iOS 14 was, uh, a requirement for, um, app users in the iOS environment to explicitly give permission, uh, for various tracking. So, um, the, the scope of the iOS 14 change was, uh, in my, in my simple brain, like just m- just much bigger.

[00:05:35] Right. A mu- much larger scope. Um, not to say that this is not impactful or, um, not potentially something that we should be concerned about or, or mindful of. Um, but just like, as far as like Tony's like ability to sleep at night, uh, the iOS 14 thing probably still gives me nightmares. Um, I, I remember it happening, like, uh, it was in September, like leading up to this time of year, um, however many years ago.

[00:05:58] I was like, "Oh my God." The scope of this doesn't feel as big. Mm-hmm. Um, yet still we are, uh, kinda like scratching at, like what, what are we-- Are we seeing any patterns in the media platforms, like conversion rate? Like are there- Mm-hmm. Maybe there's a little smoke here. Is there any fire? So, um, I'm curious if you can If you can sort of give me, uh, your POV on what, if anything, are you seeing in the data at this point in time, and, uh, what, what's, what's jumping out for you?

[00:06:28] Steve: Well, I mean, let, let's start looking at the dates too-

[00:06:31] Tony: Yeah ...

[00:06:31] Steve: with respect to this as

[00:06:32] Tony: well. Oh, yeah. That's great.

[00:06:32] Steve: That's, that's kind of a-

[00:06:33] Tony: This is an important part of the story. Sorry. Yeah. Just- I jumped ahead. I got to data. My, my apologies. Oh, yeah. So iOS, uh, 27, the dates, how is this actually rolled out into the universe?

[00:06:43] Steve: So they kind of introduced it, um, on September 14th.

[00:06:46] Tony: Yeah.

[00:06:47] Steve: So they had about a 3% adoption at that point.

[00:06:49] Tony: Yeah.

[00:06:50] Steve: But it's kind of been pressed a little bit more. In fact, I think you said on your device, you've recently, your own personal device, you've recently received, "Hey, upgrade to iOS four-- uh, 27."

[00:06:59] Tony: That's right, a push notification.

[00:07:00] Steve: Yep, a push notification on your own device recently. And it really kind of started ramping up, I think October 8th is when they officially like switched over and started pushing.

[00:07:10] Tony: From background to push notification. Yep.

[00:07:12] Steve: Exactly. Mm-hmm. It becomes like the, the recommended upgrade-

[00:07:16] Tony: Yeah ...

[00:07:16] Steve: at that point in time, instead of just also available.

[00:07:18] Tony: Yeah.

[00:07:19] Steve: So if

[00:07:19] Tony: you- Let me, let me play that back to you, just the shape of it. Mm-hmm. So September 14th, like a little over a month ago, it became available. iOS 27 was available, but you'd have to go into settings and like actively pursue the download.

[00:07:32] Steve: Right.

[00:07:32] Tony: And, and at that point in time, low adoption, three- Yeah

[00:07:35] 3% something, whatever. Yeah. Um, since mid-October, I think you said October, uh, October 8th, uh, it started to become a push notification for more and more users. My own anecdotal experience, I got the push notification. I, I did the update on the 19th. Um, the point being, like, I think maybe the way to sort of visualize it is the shape of the curve, the adoption curve is- Right

[00:07:58] probably increasing. Yeah.

[00:08:00] Steve: Yes. Exactly. It's not like a step function where all of a sudden everyone was forced to be switched over. Yeah. It's, um... I mean, Apple has a massive user base, so it's going to be a, a climb in adoption of this, um, iOS 27.

[00:08:14] Tony: Got it. Got it.

[00:08:15] Steve: Um, so a- and at the end of September, I believe the data we were looking at was around 26% of users-

[00:08:21] Tony: Okay

[00:08:21] Steve: had updated.

[00:08:22] Tony: Understood.

[00:08:22] Steve: Um, and that obviously is 21 days now, 22 days now.

[00:08:26] Tony: Mm-hmm.

[00:08:26] Steve: Since, yeah, we're, we're recording this on the 22nd, I believe.

[00:08:30] Tony: Yep.

[00:08:31] Steve: Um, so that started adopting that. Um, we didn't necessarily see on September 14th like a drop in conversion rate. Ah. Not, not immediately. Right. Um, it wasn't as we were talking about like a step function in adoption and a s- a step down in conversion.

[00:08:49] Tony: Yeah.

[00:08:49] Steve: Uh, what we did see was, uh, it was around October 5th-

[00:08:53] Tony: Interesting ...

[00:08:54] Steve: uh, uh, the decrease. That's when we started seeing a decline in conversion rate-

[00:08:57] Tony: Okay ...

[00:08:58] Steve: on Meta.

[00:08:58] Tony: Okay,

[00:08:59] Steve: cool. Um, and- Not

[00:09:00] Tony: cool, but understood.

[00:09:00] Steve: Right. Un-understandable. And then particularly around the 8th it became more apparent because it was bel- significantly below last year, and I think down like 15%, um, compared to last year from the 8th to like the 18th.

[00:09:13] Mm. So the last set of data I looked at, those 11 days, down about 15% year over year, whereas it was... Yeah, it was down before, um, like a little bit, 6% I believe.

[00:09:22] Tony: Mm-hmm.

[00:09:23] Steve: But, um, down 15% really caught our attention-

[00:09:26] Tony: Yeah ...

[00:09:27] Steve: um, in that last 11 days.

[00:09:28] Tony: Yeah. Okay. So let me, let me recap here. So playing it back, so we have, uh, iOS 27, WebKit, uh, identity browser, different pathway than iOS 14 for sure, different mechanics, both privacy related.

[00:09:42] We have the shape of the adoption, which started as a background sort of you could go do it, moved into push notification, increased the curve. Uh, and then we have-- We're beginning to see, uh, the shape of the data in, um, uh, uh- I'm sorry. The, the numbers were the y- well, the comparisons were year-on-year CVR.

[00:10:01] Steve: Mm-hmm.

[00:10:02] Tony: Uh, 15% on Meta, uh, which is like, to Steve's point, that starts to get our attention pretty rapidly. So, um, I'm curious if, um, you and I, uh, earlier this week started to explore, um, well, what, what do we see, if anything, on, on the other platforms? Was there anything that, uh, that popped out from that, from that initial analysis?

[00:10:22] Steve: Um, I really looked more at TikTok in that initial an-analysis. Yeah. And it was down slightly because that's the-- another platform we were seeing that could potentially, uh, would maybe be impacted equally- Yeah ... as rough. Um, didn't really see that much, uh, impact on Google. And Google, um, everyone defaults typically to the click date to, for the attribution of the purchase.

[00:10:46] Yeah. So it always kinda lags on conversion. So I can't tell as well on Google if the most recent, like great, the last five days have been down, but that's just because you're not getting conversions counting that'll happen in the future.

[00:10:59] Tony: Yeah.

[00:10:59] Steve: So, um, recent data always looks a little bit rougher on Google, so I, I would not-- Uh, Google data doesn't have me concerned right now.

[00:11:07] Tony: Got it.

[00:11:07] Steve: Um, and TikTok is a little bit more sparse in that we need good year-over-year comparisons for that, and we don't have as broad of our data set putting a lot of money into TikTok on a consistent basis. Yeah. Like, yeah, there, there's a solid, um... There's dozens on there, but it's not, um, as pervasive, I think.

[00:11:26] I don't know if there's any of our 200 brands that are not on Meta, 200 plus brands.

[00:11:30] Tony: Yeah, for sure. Yeah. It represents the majority of our portfolio. So let me see if I can say it back to you. So, um, the, uh, the early stage signal that we have is, um, strongest on Meta. Mm-hmm. Um, and the, the picture is less clear on Google and TikTok due to the scope of, uh, how much, how much signal we have to relate.

[00:11:49] Um, and I think, I think for me, like- I'm still kind of like, um, scratching my head around the, like the specific mechanical pathway, like the failure pathway that could present itself, um, a-and lead to. So if on one hand we're saying, "Okay, we have these changes in the, the ecosystem," right? And then on the other hand, we're saying like, "Okay, maybe we see some CVR stuff in, in Meta potentially."

[00:12:21] Steve: Yeah.

[00:12:21] Tony: I am racking my brain around like drawing a specific example of how that could be the case. Okay. So I wanna, and I wanna give like, I wanna give you my, um, my understanding of what I believe is still currently stable in the environment, uh, and then we'll go to the failure pathway. So let's say, uh, we got Jane.

[00:12:42] Jane, um, buys something, okay? Uh, Shopify records the sale, uh, and the, um, in Jane's Meta account, uh, there is a, uh, a pixel, uh, and a conversion pixel, and it's also installed on Jane's website, and then there's a CAPI integration. Shopify records the sale. The browser sends the conversion signal directly to Meta.

[00:13:03] The Shopify CAPI integration sends that same conversion signal. Um, the, the, the data matching is happening, um, the server to server on the identity pieces that are captured in Shopify and transmitted directly to, to Meta, um, including email and, um, o-other PII that's hashed between those two. Essentially that, let's call it like BAU setup or maybe like, maybe I might describe that as kinda like the best-in-class setup.

[00:13:30] Steve: Right.

[00:13:30] Tony: We good. Yes. Like there's no, there's no issue there, right?

[00:13:34] Steve: There should not be.

[00:13:34] Tony: Right.

[00:13:35] Steve: Yes.

[00:13:35] Tony: Right, right, right. Good. Thank you. I feel better now.

[00:13:38] Steve: Yeah.

[00:13:39] Tony: So, um, let's, let's sort of explore some of the potential failure paths, right? Because, um, and I'm just gonna talk this out loud and you, you gut check me and bring me back down to earth whenever, whenever I'm like, you're, you're sort of like, "No, Tony, you can't say that.

[00:13:57] That's, that's not supported by the evidence." Steve and I have this really fun relationship. Um, okay. So all right, so we had like a kinda like really good solid setup where we have Sh-Shopify. We got, um, the Meta sales channel installed. We're firing the pixel with the sales channel. We're, we're bringing CAPI in through the sales channel.

[00:14:19] Um- Really tight. Okay. Let's pretend now, uh, we have a different type of setup. We have, uh, um, whatever, I don't know, non-Shopify site, and we got a pixel, but we have no CAPI, okay? Mm-hmm. So we have no server events. Um, in this particular scenario, uh- Shopify sees the sale, the Meta pixel picks up, uh, the conversion event as well.

[00:14:51] But for whatever reason, the identity piece of that transaction is foggy, is murky. The email, whatever, it gets, it gets messed up, and there's no, there's no connection of identity. So now what's gonna happen in this scenario is Meta is going to see the sale, but it's not gonna be able to match it back to a specific ad.

[00:15:13] Yes. How, how does that sound to you?

[00:15:15] Steve: Yes. Uh, or a specific person, individual. Mm-hmm. Um, because then that's also important on the feedback loop to determine who is a good customer for your-

[00:15:24] Tony: Mm-hmm ...

[00:15:24] Steve: for

[00:15:25] Tony: your platform. So this would lean over into like, um, less... And so w- in the beginning, we sort of talked about two kind of dimensions of how this could affect you.

[00:15:32] One dimension is reporting on conversion value, like how many sales happened and what the value was. Right. Another dimension is informing targeting to the future. That example I gave you would be, like the sale would still get reported. Meta's ROAS would be s-sort of representative. Right. But the, the ability for Meta to, to sort of recognize that specific user, to archetype them, to profile them, and to expand the audience would be, would be limited.

[00:15:56] Yes?

[00:15:56] Steve: Yes.

[00:15:57] Tony: Okay. All right. Which could potentially lead to, uh, sort of kind of what you're, what you're getting at, which is the, maybe the ultimate point, AKA lower CBR fundamentally.

[00:16:08] Steve: Right.

[00:16:08] Tony: Okay, cool. All right. Yeah. I got it. So one, so one particular failure mode is, um, in, in corr- uh, uh, n-no server events, let's put it that way.

[00:16:19] Steve: Right. Yes. No server events or in some way like, um, poor server events.

[00:16:24] Tony: Yeah.

[00:16:25] Steve: Like, um, and this can sometimes manifest in different ways, especially if you have subscription-

[00:16:30] Tony: Ah ...

[00:16:31] Steve: um, or different particular purchase or conversion events that you're looking for. So that, that complicates it when it's not just a simple one-time purchase.

[00:16:39] Tony: Yeah. Yeah. Yeah, for sure. Yeah. We've had all sorts of fun puzzles around conversion, conversion- Yeah ... tracking with, uh, with subscriptions. Um, yeah, just, you know, sort of that's kind of- I'm

[00:16:51] Steve: sorry. That's what you have to deal with on a regular basis.

[00:16:52] Tony: Well, yeah, yeah. Well, Si- well, Signal has always been like... I, I've always been kind of like a ads nerd.

[00:16:57] I don't know. Um, I, I, I like Signal and Catalog and all this kind of stuff. It, it, it, it speaks to me. But, um, convert... Yeah, conversion tracking with subscription, topic for a different day. I could go on, on about that forever. So, but, but I wanted to kinda come back to something that you said, because I think this is one of the core takeaways for us, for CDC, like what we're doing in our portfolio right now is, um, this is kinda like best practice hygiene stuff like always.

[00:17:21] Um, but it's like I think particularly important when we have these big moments in time where there's tech- technology shifts in, in this, in this ecosystem. Um, it is worthwhile to talk about Shopify has, in the Shopify meta sales channel app, uh, there's different levels of integration, so, um, different settings you can choose, all the way up from maxim- maximum data sharing- Mm-hmm

[00:17:42] which essentially you can think about that as like Sending email, phone number, um, uh, uh, ad- address. Mm-hmm. All of it hashed, right? Right. So all of it, like no PII, but hashed. So, um, down to, uh, there's, there's three different settings. Uh, and so our, our position on this is, um, where, where it is applicable and appropriate for you as a brand, um, we want to share the maximum amount of information between Shopify and the Meta Pixel, and we use the Shopify app to do so in that this-- the whole point of this is to, um, ensure that this matching can happen, uh, so that we can connect a person-- a purchase to a person, and thereby have, um, really good attribution reporting in Shopify, but also really good ad targeting.

[00:18:30] So the point being is if, uh, one of the failure pathways for this iOS 27 thing to potentially have an impact on your, um, uh, your advertising, um, program, one thing to look at is your, uh, browser, uh, your Pixel and your server events, and then if you're on Shopify, I would go have a double-click into that specific data sharing setting and making sure that it's my, my recommendation set as high as you are comfortable with.

[00:18:59] Any thoughts on that?

[00:19:00] Steve: No, that, that sounds right. And, and you're far more experienced on the media side and how that should be set up.

[00:19:06] Tony: Everything. Turn on everything. Yeah. Yeah. So I think, um... Okay. So that, so that's one particular pathway. I think, uh, another thing that we do frequently as like a, a housekeeping thing, it's a simple thing to do.

[00:19:20] Uh, uh, we find surprising things all the time, and this is something I would recommend everyone does, uh, that's listening. Very simple. Look at the number of purchases that you see in Shopify. Look at the number of purchases that you see in Meta. Those numbers should be-- They don't have to equal exactly, but they shouldn't be...

[00:19:37] Here, here's an example of a situation that is obviously wrong. You have 100 purchases in Shopify, you have 200 in Meta. Okay? If you see double, it's a clue- ... uh, something's, something's goofball-y. All right? So the point being is like, um, one of the things that you want to like just stress test. In this moment in time, it's good practice just generally going into Q4- Mm-hmm

[00:20:01] um, Black Friday, Cyber Monday, but specifically post a big iOS change, go into Shopify, count your purchases for the last 30 days. Go into Meta, uh, um, the Meta Pixel and look at the number of purchases, uh, that are being counted for, um, both web Pixel events and server. The numbers should be pretty similar, yeah?

[00:20:19] Steve: Yeah.

[00:20:20] Tony: Cool.

[00:20:20] Steve: I, I think that's great.

[00:20:22] Tony: All right. Um, let's see, what else? Uh, one other thing that, um- Oh shit, what was I gonna say? Corey, cut that out.

[00:20:30] Steve: Um.

[00:20:33] Tony: Um.

[00:20:35] Steve: You got notes there for you, I think.

[00:20:37] Tony: Yeah. Uh, yeah, I think, oh shit, where was I gonna go with this?

[00:20:41] Steve: Um,

[00:20:42] Tony: so okay, so we talked about, uh, we talked about going-- we talked about, um, the potential failure paths. Okay, sorry. Corey, cut this out. So big picture, zooming out. This is not iOS 14, as best, as best as I can tell right now.

[00:20:58] Thank God. Um, this is, uh, a change to the, to the ecosystem, specifically with Apple, which always gets our attention.

[00:21:09] Steve: Mm-hmm.

[00:21:09] Tony: It's, it's a very specific, um, browser-based, um, third party, third party identity graph ecosystem kinda thing. If you are using what we would consider to be the, the best in class top tier setup, which is Shopify, sales channel, um, pixel and server events, maximized data sharing, you're gonna be in pretty good shape.

[00:21:34] Would you agree with that, Steve?

[00:21:35] Steve: I would agree. But I'd also agree with your first kinda point in that it's probably a good time to check those numbers, to audit yourself.

[00:21:42] Tony: Totally.

[00:21:43] Steve: Um, given that this change in iOS and we're in Q4 heading towards the biggest time of year for most brands.

[00:21:50] Tony: Yes. Please do an audit of your signal up and down.

[00:21:54] Uh, so do that. And if you're in this inv- if you're in this setup that I described, Shopify, sales channel apps, um, pixel and CAPI firing through the sales channel apps, maximum data sharing, you're gonna be in pretty good shape. Every sort of step away from that sort of ideal setup creates an opportunity for this particular security update to have more of an impact on your brand specifically, and I would investigate every piece of that chain that is outside of that, call it V1 setup.

[00:22:32] Um, the places that you're gonna wanna investigate, pl- the things that you're gonna wanna look at as count of purchases against count of tracked events in Meta. I would also look at your event match quality, uh, as another indicator for, uh, up and down the event stack, how, uh, how sharp is your signal? Do the audit, uh, and I think, uh, the, the sort of fundamental takeaway for me at this moment in time is check, check the plumbing before blaming everything on Apple.

[00:23:11] Uh, we-- I think we have something worth investigating. We're gonna keep pulling on it. Um, but I don't, I don't think... I-- At this point in time, I don't feel like this is an, an explanation for, like, g-global falling conversion rates across the, across the media stack. It is something that we wanna keep, keep pulling on.

[00:23:30] Um, but, uh, yeah. Yeah. That's, that's my, my two cents. What, what's your takeaway?

[00:23:36] Steve: Yeah. I mean, we are all also just after Amazon Prime Day. Sure. So, like, I kinda almost expect a little bit of drop-off, and October always has its rough bumps. Um, like, it, it's a tougher time of year for businesses. But it is a time to kinda straighten this out, um, make sure everything is correct.

[00:23:54] Um, we could potentially run an incrementality test.

[00:23:58] Tony: Uh,

[00:23:58] Steve: yes. That, that was the other thing we talked about. That's

[00:24:00] Tony: right.

[00:24:01] Steve: But d-there might be a strong impetus to hold off until January, because we're at, like, 26% adoption at the end of September. Maybe we're a little bit higher now. It might, you know. But I don't think full adoption has happened yet, so if you do come up with a number now, um, it's not indicative of how it would look with everyone on Apple being on O- iOS 27.

[00:24:25] Yes. That'll probably happen by early January.

[00:24:27] Tony: Yes.

[00:24:27] Steve: Like, early next year is what Apple was talking about.

[00:24:29] Tony: Yes. Well, yeah. And the Achilles heel of all incrementality testing, which is something I believe in deeply, uh, is it's a point in time measurement. Uh- Right. So, I mean, I think my, my philosophy on this is, like, at, at all times you should be running a geo holdout test.

[00:24:47] I think you and I, uh, are having this friendly debate, uh, ongoing. Like, I, I wanna run sequential, uh, not, not... What's the opposite of sequential? Concurrent. Con-concurrent tests.

[00:24:59] Steve: Right.

[00:24:59] Tony: And Steve's like, "No, you can't do that, bro."

[00:25:01] Steve: So- Well, I mean, we can, it just requires a, a lot more work and a-and a lot more, um, thoroughness to make sure that there's not a kind of crossover impact- Yeah

[00:25:12] happening from one test to another.

[00:25:13] Tony: Well, yeah, yeah, yeah. Tra-- yeah. Thank you, Steve, for keep-keeping my, uh, tasks clean. Yeah. But let me say it back to you what, what you're saying here, so which is, uh, we- We can and will, uh, we're always, uh, eye testing. Um, we, the, the-- We just had a conversation about this yesterday.

[00:25:28] The vast, the, uh, large collection of-- The largest portion of eye test we have is around Meta.

[00:25:33] Steve: Mm-hmm.

[00:25:33] Tony: Um, that testing is concentrated into seven-day click attribution, um, ROAS bidding, and, uh, we're, we're always looking at, um, expanding and deepening. I would say broadening and deepening our, uh, our benchmarks, our, our eye test factors around Meta, and I think this iOS, iOS changes are a really good, really good point for us to kinda like bring that topic up again, so.

[00:25:59] Steve: Right.

[00:25:59] Tony: We'll be off to the races and resetting that, and we'll, uh, we'll come back to the, to the pod here and let, let everybody know if we find anything material. Um, but, uh, yeah. Other than that, listen, check your signal. I know it's boring, okay? But go, go, go do this. This is the simplest thing. Anybody can do this.

[00:26:16] Everyone can do this. Go count your Shopify purchases for the last 30 days. You got that. Go into Meta, uh, go into the event page, look at the purchase count, uh, pixel, and server. Uh, make sure those numbers are pretty close. Anything else?

[00:26:33] Steve: No. I mean, that's what our teams are doing today.

[00:26:36] Tony: That's what we're up to.

[00:26:37] Steve: Yeah.

[00:26:37] Tony: All right. Happy hunting, y'all. Fourth quarter, carry on.