How to Check Your Ad Attribution Against Post-Purchase Survey Answers on Shopify

TL;DR

Survey totals and attribution totals can agree while the orders underneath them don't. The useful check runs order by order: what the buyer said, what Shopify recorded for that order's visits, and what GA4 recorded for the same transaction. Before Black Friday, confirm where your survey app keeps its answers, since a Thank you page survey can't write to the order from the page [1]. Afterwards, drop returning buyers and build the matrix.

Key Takeaways

• Key every answer to the order's ID, the number in its admin URL. Shopify's docs say the integer behind a name like #1001 isn't guaranteed unique, even within one store [9].
• Ask your survey app where answers are stored. Shopify's own tutorial says to keep them "in a database, or in metafields" [2].
• Read the conversion summary's first and last visit for each answered order, under Orders → the order → Conversion summary → View conversion details [7].
• Split returning customers out before counting. Their answers describe a habit.
• With 300 answers, a 30% channel share carries about 5 points of error either way (illustrative arithmetic below).
• Install or change the survey before the November 13 app stop on the BFCM readiness hub.

Start from the post-purchase survey matrix you want in December

It helps to see the finished thing first. Here's an illustrative store: 200 first-time buyers answered "How did you hear about us?" after Black Friday, and each answer sits next to the source Shopify recorded for that order's first visit.

Read it as channel totals and Meta looks settled: 62 buyers name Instagram or Facebook, and Shopify credits 58 first visits to Meta. Within four orders. Read the rows instead and only 38 orders agree. Fourteen people who arrived first from a Meta link say they found you on Google, and thirteen who name Instagram show up in Shopify as Direct.

The totals matched by accident. Two different groups of 20-odd orders cancelled each other out, and the pie chart would never have shown you either one.

You've probably had a week where the survey and the Meta dashboard told two different stories about the same month. The matrix shows which orders the stories differ on. The rest is getting three columns onto one row per order, and reading the off-diagonal cells.

Where do survey answers live after the Thank you page upgrade?

Post-purchase surveys used to run as scripts on the old order status page; our thank you page deadline article covers why they became checkout UI extensions. What matters here is where the new ones can put an answer.

Shopify lists "Post-purchase survey" as a use case for the Thank you page block target, then says "Write access isn't available on Thank you page targets because the order is complete" [1]. Its own survey tutorial, built on the Thank you and Order status page targets [3], adds: "In a production-ready application, you should store the survey response in a database, or in metafields" [2].

So the answer goes wherever the survey app's backend sends it, which leaves three cases:

A question asked inside checkout, before payment, can write cart attributes or cart metafields, and Shopify recommends metafields [4]. Cart metafields only reach the order if a matching order metafield definition is set to copy from cart to order [5]. Shopify also notes that "Attribute changes aren't available when the buyer uses an accelerated checkout method such as Apple Pay or Google Pay" [4], so an in-checkout question can miss your express-checkout buyers entirely.

If your survey app can't export answers with an order identifier, better to learn it in October.

Three columns on one order

Each row of the join has an order ID, a survey answer, and two recorded sources.

Shopify's column. The conversion summary on each order shows "The origin of the customer's first visit to your store" [7], and Shopify's API names the visits firstVisit and lastVisit [8]. When cookies were blocked, the summary is limited, with no timeline and no View conversion details [7]. Details can take "up to 48 hours to display" [7], so build the sheet a few days after the orders.

The admin shows the journey one order at a time. For a week of answers, that's a manageable afternoon; for all of November, have someone with Admin API access pull customerJourneySummary for each order instead [8].

GA4's column. In GA4, go to Explore → Free form, add Transaction ID as the row, and add Session source / medium and First user source / medium beside it. Those are the dimensions transactionId, sessionSourceMedium and firstUserSourceMedium in Google's schema [10]. On a store running WeltPixel Conversion Tracking, the Transaction ID GA4 shows for a purchase is the order's ID from its admin URL, not the #1001 order name. That's why the order ID is the join key on every row.

Ad click IDs don't belong in this sheet, because Shopify doesn't display them on the order; our guide to checking whether ad click IDs survive to your orders covers what an order will and won't tell you. How Shopify's attribution model works traces a single order this way. The matrix is that trace done a few hundred times, with the buyer's answer as a third column.

What does each disagreement usually mean?

Most disagreement cells have a structural cause you can name before blaming either source. Treat these as starting points and check a few orders per cell by hand.

The recall row has a source outside ecommerce. In a 2018 peer-reviewed study of a health campaign run across TV, online video and online display, 64% of 719 adults said they'd seen it, and of those, 91% said on television, 8% online video and 21% online display [11]. The authors found that respondents "assumed their exposure had occurred via television, regardless of the actual source of exposure" [11]. That was a public health campaign, not a store, so read the direction from it.

Untagged email and SMS links land in the Direct column too; our UTM tagging rules fix those.

How many answers before a pattern counts?

More than you'd guess, and per cell, more than you'll have. Run the arithmetic on the same illustrative store: 1,000 Black Friday orders and 300 survey answers. If 30% of answers name one channel, the 95% margin of error on that share is about 5 points either way, assuming the people who answer are a random sample of your buyers (illustrative arithmetic: 1.96 × √(0.3 × 0.7 ÷ 300) ≈ 0.052). They aren't quite, since some buyers skip surveys more than others, so read 5 points as the best case.

Then split out returning customers. Thank you page extensions can read whether this is "the customer's first completed order with this shop" [12], so ask whether your survey app stores that with the answer. In the illustration, 200 of the 300 answers came from first-time buyers, and the margin on a 31% share of those 200 (the 62 Instagram or Facebook answers) widens to about 6 points.

Cells are where it gets thin. The cell that would suggest Meta is under-credited, Instagram answers that Shopify files as Direct, holds 13 orders. That's enough to open each order and look. It isn't enough to move a budget.

We couldn't find a neutral, published benchmark for survey response rates; the figures we found were from survey vendors and unsourced. Your response rate is your own number, and it decides how many weeks of answers you need before the cells fill.

A survey also can't tell you whether an ad caused the sale. For cause, a Meta ads incrementality test is the tool. For channel totals across the whole store, Shopify's channel performance report is.

Set it up before November 13

1. In your survey app, find where answers are stored and what its export contains. You need the order ID, or at least the order name, on every row.
2. Install a survey app or move its placement before November 13, the app stop on the Black Friday tracking freeze calendar.
3. Keep the answer options fixed through the season, then place one test order and confirm the answer reaches the export with the right order identifier.
4. One week after Black Friday, export that week's answers. Drop returning customers.
5. For each answered order, record the conversion summary's first and last visit [7]. Then export GA4's Transaction ID, session source / medium and first user source / medium from an exploration [10].
6. Count the pairs into the matrix. Open three orders from each off-diagonal cell and check whether the usual reading in the table above holds.
7. Decide only on cells that repeat over at least two weeks of answers.

FAQ

Is a post-purchase survey more accurate than Shopify's attribution?

Neither is the truth. A survey records what the buyer remembers, and Shopify records visits within a window of at most 30 days [8]. Read them side by side, order by order.

Can a Thank you page survey save the answer to the order?

Not from the page itself: Thank you page targets have no write access because the order is complete [1]. The survey app's backend stores the answer, in its own database or in metafields [2].

Does WeltPixel Conversion Tracking read survey answers?

No. The app doesn't read, store or forward survey answers, order notes or anything another app attaches to an order. The survey and the attribution stay separate columns, which is what makes the comparison worth running.

What question should a Shopify attribution survey ask?

One question, asked of every buyer, in the "How did you hear about us?" shape, with fixed answer options that match the channels you can see in Shopify and GA4. Add "Other" with a text box and read those by hand.

Keep the GA4 column keyed to the order

WeltPixel Conversion Tracking sends each storefront purchase to GA4 server-side with the order's ID as the Transaction ID, so your survey check's GA4 column lines up with the order. GA4 is on the free plan, up to 100 storefront orders a month; Plus is $39/month with no order limit and also sends admin and POS orders [14].

Sources

1. Shopify developer documentation, checkout UI extensions, Thank you page block target (purchase.thank-you.block.render), shopify.dev/docs/api/checkout-ui-extensions/latest/targets/thank-you/block, accessed October 13, 2026
2. Shopify developer documentation, "Add a survey to Thank you and Order status pages", shopify.dev/docs/apps/build/checkout/thank-you-order-status/add-survey, accessed October 13, 2026
3. Shopify developer documentation, customer account UI extension targets (Order status page), shopify.dev/docs/api/customer-account-ui-extensions/latest/targets, accessed October 13, 2026
4. Shopify developer documentation, checkout UI extensions Attributes API, shopify.dev/docs/api/checkout-ui-extensions/latest/target-apis/checkout-apis/attributes-api, accessed October 13, 2026
5. Shopify developer documentation, checkout UI extensions Metafields API, shopify.dev/docs/api/checkout-ui-extensions/latest/target-apis/platform-apis/metafields-api, accessed October 13, 2026
6. Shopify developer documentation, "Metafields" in checkout (app-owned metafields), shopify.dev/docs/apps/build/checkout/metafields, accessed October 13, 2026
7. Shopify Help Center, "Conversion summary", help.shopify.com/en/manual/fulfillment/managing-orders/analytics/conversion-summary, accessed October 13, 2026
8. Shopify developer documentation, GraphQL Admin API CustomerJourneySummary object, shopify.dev/docs/api/admin-graphql/latest/objects/CustomerJourneySummary, accessed October 13, 2026
9. Shopify developer documentation, GraphQL Admin API Order object (name, number), shopify.dev/docs/api/admin-graphql/latest/objects/Order, accessed October 13, 2026
10. Google Analytics Data API, "API Dimensions & Metrics", developers.google.com/analytics/devguides/reporting/data/v1/api-schema, accessed October 13, 2026
11. Pettigrew S, Jongenelis M, Phillips F, Slevin T, Allom V, Keightley S, Beasley S, "Assessing Audience Members' Ability to Identify the Media Source of a Health Campaign Disseminated via Different Media", Frontiers in Public Health, July 19, 2018, frontiersin.org/journals/public-health/articles/10.3389/fpubh.2018.00196/full, accessed October 13, 2026
12. Shopify developer documentation, checkout UI extensions Order API (isFirstOrder), shopify.dev/docs/api/checkout-ui-extensions/latest/target-apis/checkout-apis/order-api, accessed October 13, 2026
13. Shopify Help Center, "Exporting orders" (Note Attributes column), help.shopify.com/en/manual/fulfillment/managing-orders/exporting-orders, accessed October 13, 2026
14. WeltPixel Conversion Tracking, Shopify App Store listing (free plan GA4 up to 100 orders a month; Plus $39/month), apps.shopify.com/weltpixel-conversion-tracking, accessed October 13, 2026