Three real-world scenarios where raw intuition fails. See why organizations rely on statistical thinking to extract signal from noise.
Statistics is not about memorizing formulas; it is about asking the right questions of data. Each scenario below presents raw data and an intuitive conclusion. Your job is to decide whether to trust it.
A company launches a new feature. After 1 week, 45% of users adopted it. Marketing says 'success!' but the data analyst is skeptical. The raw data covers 200 users with adoption status, device type, traffic source, and user tenure.
Raw Data
| User ID | Device | Traffic Source | Tenure (wks) | Adopted |
|---|---|---|---|---|
| U0001 | Desktop | Paid | 38 | No |
| U0002 | Desktop | Organic | 50 | No |
| U0003 | Mobile | Organic | 5 | Yes |
| U0004 | Mobile | Paid | 12 | No |
| U0005 | Desktop | Organic | 5 | Yes |
| U0006 | Mobile | Referral | 38 | No |
| U0007 | Mobile | Organic | 44 | Yes |
| U0008 | Desktop | Paid | 12 | No |
| U0009 | Desktop | Referral | 47 | Yes |
| U0010 | Mobile | Referral | 32 | No |
| U0011 | Desktop | Organic | 18 | No |
| U0012 | Desktop | Organic | 27 | No |
| U0013 | Desktop | Organic | 10 | No |
| U0014 | Mobile | Paid | 4 | Yes |
| U0015 | Desktop | Referral | 9 | No |
| U0016 | Mobile | Organic | 35 | Yes |
| U0017 | Desktop | Referral | 9 | Yes |
| U0018 | Mobile | Organic | 1 | No |
| U0019 | Mobile | Paid | 12 | Yes |
| U0020 | Mobile | Organic | 51 | No |
| U0021 | Mobile | Organic | 34 | Yes |
| U0022 | Desktop | Paid | 47 | No |
| U0023 | Mobile | Organic | 41 | Yes |
| U0024 | Mobile | Paid | 42 | Yes |
| U0025 | Desktop | Paid | 16 | No |
| U0026 | Mobile | Paid | 3 | Yes |
| U0027 | Desktop | Organic | 36 | No |
| U0028 | Desktop | Referral | 6 | No |
| U0029 | Desktop | Paid | 12 | No |
| U0030 | Mobile | Referral | 0 | Yes |
| U0031 | Mobile | Paid | 6 | Yes |
| U0032 | Desktop | Referral | 20 | No |
| U0033 | Mobile | Paid | 13 | Yes |
| U0034 | Desktop | Paid | 10 | No |
| U0035 | Mobile | Organic | 34 | Yes |
| U0036 | Mobile | Organic | 18 | No |
| U0037 | Desktop | Organic | 49 | No |
| U0038 | Mobile | Paid | 25 | Yes |
| U0039 | Desktop | Paid | 21 | No |
| U0040 | Desktop | Paid | 5 | No |
| U0041 | Desktop | Organic | 18 | No |
| U0042 | Desktop | Referral | 47 | Yes |
| U0043 | Mobile | Organic | 26 | No |
| U0044 | Mobile | Referral | 51 | No |
| U0045 | Mobile | Referral | 46 | No |
| U0046 | Mobile | Referral | 50 | No |
| U0047 | Desktop | Organic | 28 | No |
| U0048 | Mobile | Organic | 41 | No |
| U0049 | Desktop | Paid | 38 | No |
| U0050 | Mobile | Paid | 23 | Yes |
| U0051 | Mobile | Paid | 23 | No |
| U0052 | Mobile | Organic | 29 | Yes |
| U0053 | Desktop | Paid | 11 | Yes |
| U0054 | Desktop | Paid | 26 | No |
| U0055 | Mobile | Referral | 7 | Yes |
| U0056 | Desktop | Organic | 23 | No |
| U0057 | Mobile | Paid | 2 | Yes |
| U0058 | Desktop | Paid | 19 | No |
| U0059 | Mobile | Referral | 47 | Yes |
| U0060 | Desktop | Referral | 13 | No |
| U0061 | Desktop | Paid | 33 | Yes |
| U0062 | Desktop | Referral | 50 | No |
| U0063 | Desktop | Referral | 1 | No |
| U0064 | Desktop | Paid | 21 | No |
| U0065 | Mobile | Paid | 32 | No |
| U0066 | Mobile | Referral | 50 | Yes |
| U0067 | Desktop | Paid | 51 | Yes |
| U0068 | Desktop | Organic | 51 | Yes |
| U0069 | Desktop | Organic | 3 | Yes |
| U0070 | Mobile | Referral | 38 | Yes |
| U0071 | Desktop | Paid | 2 | No |
| U0072 | Desktop | Paid | 38 | No |
| U0073 | Desktop | Paid | 41 | No |
| U0074 | Desktop | Referral | 42 | No |
| U0075 | Desktop | Paid | 12 | No |
| U0076 | Desktop | Paid | 6 | No |
| U0077 | Desktop | Organic | 27 | No |
| U0078 | Desktop | Referral | 1 | No |
| U0079 | Desktop | Referral | 43 | No |
| U0080 | Desktop | Referral | 19 | No |
| U0081 | Desktop | Paid | 1 | No |
| U0082 | Mobile | Organic | 2 | Yes |
| U0083 | Mobile | Organic | 45 | Yes |
| U0084 | Mobile | Referral | 51 | No |
| U0085 | Mobile | Organic | 27 | Yes |
| U0086 | Desktop | Paid | 30 | No |
| U0087 | Desktop | Referral | 18 | No |
| U0088 | Mobile | Paid | 37 | Yes |
| U0089 | Mobile | Paid | 22 | No |
| U0090 | Desktop | Paid | 39 | Yes |
| U0091 | Mobile | Paid | 26 | No |
| U0092 | Mobile | Paid | 10 | Yes |
| U0093 | Desktop | Paid | 7 | No |
| U0094 | Desktop | Referral | 39 | No |
| U0095 | Mobile | Paid | 24 | Yes |
| U0096 | Mobile | Organic | 38 | Yes |
| U0097 | Desktop | Organic | 42 | No |
| U0098 | Mobile | Referral | 46 | Yes |
| U0099 | Desktop | Organic | 29 | Yes |
| U0100 | Mobile | Referral | 19 | No |
| U0101 | Desktop | Organic | 35 | Yes |
| U0102 | Mobile | Referral | 13 | Yes |
| U0103 | Mobile | Organic | 20 | Yes |
| U0104 | Mobile | Paid | 13 | No |
| U0105 | Mobile | Referral | 20 | Yes |
| U0106 | Desktop | Organic | 10 | Yes |
| U0107 | Mobile | Organic | 0 | No |
| U0108 | Desktop | Organic | 31 | Yes |
| U0109 | Desktop | Paid | 40 | No |
| U0110 | Mobile | Organic | 44 | Yes |
| U0111 | Desktop | Organic | 27 | No |
| U0112 | Desktop | Organic | 45 | Yes |
| U0113 | Mobile | Referral | 27 | No |
| U0114 | Desktop | Organic | 51 | No |
| U0115 | Mobile | Referral | 29 | Yes |
| U0116 | Desktop | Referral | 14 | Yes |
| U0117 | Desktop | Referral | 3 | Yes |
| U0118 | Mobile | Referral | 40 | Yes |
| U0119 | Desktop | Paid | 9 | No |
| U0120 | Desktop | Referral | 39 | No |
| U0121 | Desktop | Referral | 12 | No |
| U0122 | Mobile | Organic | 11 | Yes |
| U0123 | Desktop | Organic | 51 | No |
| U0124 | Desktop | Paid | 40 | No |
| U0125 | Mobile | Organic | 32 | Yes |
| U0126 | Mobile | Paid | 22 | Yes |
| U0127 | Desktop | Referral | 13 | No |
| U0128 | Desktop | Referral | 12 | Yes |
| U0129 | Desktop | Organic | 20 | Yes |
| U0130 | Mobile | Organic | 44 | No |
| U0131 | Mobile | Referral | 21 | Yes |
| U0132 | Mobile | Organic | 33 | Yes |
| U0133 | Mobile | Organic | 32 | No |
| U0134 | Mobile | Paid | 27 | Yes |
| U0135 | Mobile | Organic | 5 | Yes |
| U0136 | Mobile | Paid | 18 | Yes |
| U0137 | Mobile | Referral | 50 | No |
| U0138 | Mobile | Referral | 7 | Yes |
| U0139 | Desktop | Organic | 22 | No |
| U0140 | Mobile | Organic | 45 | No |
| U0141 | Desktop | Referral | 43 | No |
| U0142 | Desktop | Organic | 35 | No |
| U0143 | Desktop | Organic | 12 | Yes |
| U0144 | Desktop | Organic | 20 | No |
| U0145 | Mobile | Paid | 33 | No |
| U0146 | Desktop | Referral | 44 | Yes |
| U0147 | Desktop | Paid | 22 | No |
| U0148 | Desktop | Paid | 35 | No |
| U0149 | Desktop | Paid | 35 | No |
| U0150 | Mobile | Referral | 35 | Yes |
| U0151 | Mobile | Referral | 22 | No |
| U0152 | Mobile | Referral | 1 | No |
| U0153 | Mobile | Referral | 20 | Yes |
| U0154 | Mobile | Referral | 46 | Yes |
| U0155 | Desktop | Referral | 16 | No |
| U0156 | Desktop | Organic | 34 | No |
| U0157 | Mobile | Organic | 32 | Yes |
| U0158 | Mobile | Referral | 38 | Yes |
| U0159 | Desktop | Referral | 24 | No |
| U0160 | Mobile | Paid | 4 | No |
| U0161 | Mobile | Referral | 34 | No |
| U0162 | Desktop | Organic | 38 | No |
| U0163 | Mobile | Referral | 25 | Yes |
| U0164 | Mobile | Paid | 32 | No |
| U0165 | Mobile | Referral | 30 | Yes |
| U0166 | Mobile | Organic | 49 | No |
| U0167 | Mobile | Organic | 9 | Yes |
| U0168 | Mobile | Referral | 35 | No |
| U0169 | Mobile | Paid | 1 | No |
| U0170 | Desktop | Organic | 50 | No |
| U0171 | Mobile | Referral | 2 | No |
| U0172 | Desktop | Referral | 31 | No |
| U0173 | Desktop | Referral | 31 | Yes |
| U0174 | Mobile | Organic | 24 | No |
| U0175 | Desktop | Organic | 4 | No |
| U0176 | Mobile | Organic | 41 | No |
| U0177 | Mobile | Paid | 18 | No |
| U0178 | Mobile | Organic | 8 | Yes |
| U0179 | Desktop | Paid | 15 | No |
| U0180 | Desktop | Organic | 4 | No |
| U0181 | Desktop | Paid | 28 | No |
| U0182 | Desktop | Organic | 6 | No |
| U0183 | Desktop | Referral | 22 | No |
| U0184 | Desktop | Referral | 28 | No |
| U0185 | Desktop | Referral | 19 | No |
| U0186 | Mobile | Paid | 32 | Yes |
| U0187 | Desktop | Referral | 1 | Yes |
| U0188 | Mobile | Paid | 10 | No |
| U0189 | Mobile | Organic | 37 | No |
| U0190 | Desktop | Organic | 25 | No |
| U0191 | Desktop | Paid | 9 | No |
| U0192 | Mobile | Paid | 24 | Yes |
| U0193 | Desktop | Referral | 17 | No |
| U0194 | Desktop | Paid | 42 | Yes |
| U0195 | Desktop | Paid | 22 | Yes |
| U0196 | Mobile | Referral | 22 | No |
| U0197 | Mobile | Referral | 6 | Yes |
| U0198 | Desktop | Paid | 49 | No |
| U0199 | Desktop | Referral | 32 | Yes |
| U0200 | Desktop | Organic | 2 | No |
Can we confidently say the feature is a success based on 45% adoption?
You are auditing a loan approval algorithm. In the raw data: 70% of Group A loans defaulted; 30% of Group B loans defaulted. The conclusion: 'Group A is riskier.' But there is hidden structure: Group A has a different income composition than Group B.
Raw Data
| Loan ID | Group | Income Level | Defaulted |
|---|---|---|---|
| L0001 | A | High | No |
| L0002 | A | High | Yes |
| L0003 | A | High | No |
| L0004 | A | High | No |
| L0005 | A | High | No |
| L0006 | A | High | No |
| L0007 | A | High | No |
| L0008 | A | High | No |
| L0009 | A | High | No |
| L0010 | A | High | No |
| L0011 | A | High | No |
| L0012 | A | High | No |
| L0013 | A | High | No |
| L0014 | A | High | No |
| L0015 | A | High | No |
| L0016 | A | High | No |
| L0017 | A | High | No |
| L0018 | A | High | No |
| L0019 | A | High | No |
| L0020 | A | High | No |
| L0021 | A | High | Yes |
| L0022 | A | High | No |
| L0023 | A | High | No |
| L0024 | A | High | No |
| L0025 | A | High | Yes |
| L0026 | A | High | No |
| L0027 | A | High | No |
| L0028 | A | High | No |
| L0029 | A | High | Yes |
| L0030 | A | High | Yes |
| L0031 | A | High | No |
| L0032 | A | High | No |
| L0033 | A | High | No |
| L0034 | A | High | No |
| L0035 | A | High | No |
| L0036 | A | High | No |
| L0037 | A | High | No |
| L0038 | A | High | No |
| L0039 | A | High | Yes |
| L0040 | A | High | No |
| L0041 | A | High | No |
| L0042 | A | High | Yes |
| L0043 | A | High | No |
| L0044 | A | High | No |
| L0045 | A | High | No |
| L0046 | A | High | No |
| L0047 | A | High | No |
| L0048 | A | High | No |
| L0049 | A | High | Yes |
| L0050 | A | High | No |
| L0051 | A | Low | Yes |
| L0052 | A | Low | Yes |
| L0053 | A | Low | Yes |
| L0054 | A | Low | Yes |
| L0055 | A | Low | Yes |
| L0056 | A | Low | No |
| L0057 | A | Low | Yes |
| L0058 | A | Low | No |
| L0059 | A | Low | Yes |
| L0060 | A | Low | Yes |
| L0061 | A | Low | Yes |
| L0062 | A | Low | Yes |
| L0063 | A | Low | Yes |
| L0064 | A | Low | No |
| L0065 | A | Low | Yes |
| L0066 | A | Low | Yes |
| L0067 | A | Low | Yes |
| L0068 | A | Low | Yes |
| L0069 | A | Low | Yes |
| L0070 | A | Low | Yes |
| L0071 | A | Low | Yes |
| L0072 | A | Low | Yes |
| L0073 | A | Low | Yes |
| L0074 | A | Low | Yes |
| L0075 | A | Low | Yes |
| L0076 | A | Low | No |
| L0077 | A | Low | Yes |
| L0078 | A | Low | Yes |
| L0079 | A | Low | Yes |
| L0080 | A | Low | Yes |
| L0081 | A | Low | Yes |
| L0082 | A | Low | Yes |
| L0083 | A | Low | No |
| L0084 | A | Low | Yes |
| L0085 | A | Low | No |
| L0086 | A | Low | Yes |
| L0087 | A | Low | No |
| L0088 | A | Low | No |
| L0089 | A | Low | Yes |
| L0090 | A | Low | No |
| L0091 | A | Low | Yes |
| L0092 | A | Low | No |
| L0093 | A | Low | Yes |
| L0094 | A | Low | Yes |
| L0095 | A | Low | No |
| L0096 | A | Low | Yes |
| L0097 | A | Low | Yes |
| L0098 | A | Low | Yes |
| L0099 | A | Low | Yes |
| L0100 | A | Low | Yes |
| L0101 | B | High | No |
| L0102 | B | High | No |
| L0103 | B | High | No |
| L0104 | B | High | No |
| L0105 | B | High | No |
| L0106 | B | High | No |
| L0107 | B | High | No |
| L0108 | B | High | No |
| L0109 | B | High | No |
| L0110 | B | High | No |
| L0111 | B | High | No |
| L0112 | B | High | No |
| L0113 | B | High | Yes |
| L0114 | B | High | No |
| L0115 | B | High | No |
| L0116 | B | High | Yes |
| L0117 | B | High | No |
| L0118 | B | High | No |
| L0119 | B | High | No |
| L0120 | B | High | Yes |
| L0121 | B | High | No |
| L0122 | B | High | Yes |
| L0123 | B | High | No |
| L0124 | B | High | Yes |
| L0125 | B | High | No |
| L0126 | B | High | Yes |
| L0127 | B | High | No |
| L0128 | B | High | No |
| L0129 | B | High | No |
| L0130 | B | High | No |
| L0131 | B | High | No |
| L0132 | B | High | No |
| L0133 | B | High | No |
| L0134 | B | High | No |
| L0135 | B | High | No |
| L0136 | B | High | No |
| L0137 | B | High | No |
| L0138 | B | High | No |
| L0139 | B | High | No |
| L0140 | B | High | Yes |
| L0141 | B | High | No |
| L0142 | B | High | No |
| L0143 | B | High | Yes |
| L0144 | B | High | No |
| L0145 | B | High | Yes |
| L0146 | B | High | No |
| L0147 | B | High | Yes |
| L0148 | B | High | Yes |
| L0149 | B | High | No |
| L0150 | B | High | No |
| L0151 | B | High | No |
| L0152 | B | High | No |
| L0153 | B | High | Yes |
| L0154 | B | High | No |
| L0155 | B | High | No |
| L0156 | B | High | No |
| L0157 | B | High | Yes |
| L0158 | B | High | No |
| L0159 | B | High | No |
| L0160 | B | High | No |
| L0161 | B | High | No |
| L0162 | B | High | No |
| L0163 | B | High | No |
| L0164 | B | High | No |
| L0165 | B | High | No |
| L0166 | B | High | No |
| L0167 | B | High | Yes |
| L0168 | B | High | Yes |
| L0169 | B | High | No |
| L0170 | B | High | No |
| L0171 | B | High | Yes |
| L0172 | B | High | No |
| L0173 | B | High | No |
| L0174 | B | High | No |
| L0175 | B | High | No |
| L0176 | B | High | No |
| L0177 | B | High | No |
| L0178 | B | High | Yes |
| L0179 | B | High | No |
| L0180 | B | High | No |
| L0181 | B | Low | Yes |
| L0182 | B | Low | No |
| L0183 | B | Low | Yes |
| L0184 | B | Low | Yes |
| L0185 | B | Low | Yes |
| L0186 | B | Low | Yes |
| L0187 | B | Low | Yes |
| L0188 | B | Low | Yes |
| L0189 | B | Low | Yes |
| L0190 | B | Low | No |
| L0191 | B | Low | No |
| L0192 | B | Low | Yes |
| L0193 | B | Low | No |
| L0194 | B | Low | Yes |
| L0195 | B | Low | No |
| L0196 | B | Low | No |
| L0197 | B | Low | Yes |
| L0198 | B | Low | Yes |
| L0199 | B | Low | No |
| L0200 | B | Low | Yes |
Is Group A truly riskier, or is something else happening?
A pharmaceutical company reports their new drug reduced symptom severity by an average of 8 points (on a 100-point scale) compared to a placebo (which showed 2 points improvement). The company claims 'clinically meaningful.' You see all individual data points in the raw table.
Raw Data
| Subject ID | Group | Score Before | Score After |
|---|---|---|---|
| T001 | Treated | 40 | 45 |
| T002 | Treated | 72 | 66 |
| T003 | Treated | 69 | 55 |
| T004 | Treated | 66 | 66 |
| T005 | Treated | 50 | 40 |
| T006 | Treated | 78 | 71 |
| T007 | Treated | 50 | 43 |
| T008 | Treated | 51 | 44 |
| T009 | Treated | 57 | 40 |
| T010 | Treated | 78 | 66 |
| T011 | Treated | 53 | 29 |
| T012 | Treated | 52 | 55 |
| T013 | Treated | 70 | 71 |
| T014 | Treated | 44 | 28 |
| T015 | Treated | 47 | 41 |
| T016 | Treated | 44 | 36 |
| T017 | Treated | 79 | 64 |
| T018 | Treated | 74 | 65 |
| T019 | Treated | 65 | 45 |
| T020 | Treated | 44 | 45 |
| T021 | Treated | 58 | 56 |
| T022 | Treated | 61 | 50 |
| T023 | Treated | 60 | 60 |
| T024 | Treated | 61 | 50 |
| T025 | Treated | 79 | 79 |
| T026 | Treated | 75 | 72 |
| T027 | Treated | 61 | 61 |
| T028 | Treated | 62 | 54 |
| T029 | Treated | 46 | 26 |
| T030 | Treated | 40 | 16 |
| C001 | Control | 62 | 47 |
| C002 | Control | 44 | 39 |
| C003 | Control | 67 | 53 |
| C004 | Control | 53 | 37 |
| C005 | Control | 73 | 76 |
| C006 | Control | 62 | 72 |
| C007 | Control | 57 | 55 |
| C008 | Control | 57 | 41 |
| C009 | Control | 46 | 30 |
| C010 | Control | 55 | 59 |
| C011 | Control | 78 | 78 |
| C012 | Control | 62 | 69 |
| C013 | Control | 78 | 80 |
| C014 | Control | 75 | 63 |
| C015 | Control | 77 | 61 |
| C016 | Control | 43 | 45 |
| C017 | Control | 70 | 58 |
| C018 | Control | 60 | 50 |
| C019 | Control | 71 | 73 |
| C020 | Control | 66 | 74 |
| C021 | Control | 47 | 49 |
| C022 | Control | 57 | 50 |
| C023 | Control | 48 | 43 |
| C024 | Control | 57 | 61 |
| C025 | Control | 76 | 77 |
| C026 | Control | 63 | 72 |
| C027 | Control | 76 | 82 |
| C028 | Control | 74 | 75 |
| C029 | Control | 72 | 81 |
| C030 | Control | 70 | 56 |
Is the drug's 8-point effect real and clinically meaningful, or could it be noise?