Subscribe to 'Vogue' on YouTube to keep up with all of our latest videos and shows.
0<\alpha\leq\alpha_{n}\leq\beta<1 ( \alpha, \beta\in (0,1) );
\liminf_{n\rightarrow\infty}r_{n}>0 \lim_{n\rightarrow\infty }|r_{n+1}-r_{n}|=0 .
\{(x_{n}, y_{n})\}
(
Lonsdale Southwick Grey Womens Sneakers Size 10 Ll9nycGSDh
).
, , -, \{ (x_{n},y_{n})\} ( 1.10 ).
The variational inequality problem (VIP) is formulated as the problem of finding a point x^{*} with property x^{*}\in C , \langle Ax^{*},z-x^{*}\rangle\geq0 , \forall z\in C . We will denote the solution set of VIP by VI(A, C) .
A mapping A:C\rightarrow H is said to be an α -inverse-strongly monotone mapping if there exists a constant \alpha>0 such that \langle Ax-Ay,x-y\rangle\geq\alpha\|Ax-Ay\|^{2} for any x,y\in C . Setting F(x,y)=\langle Ax,y-x\rangle , it is easy to show that F satisfies conditions (A1)-(A4) as A is an α -inverse-strongly monotone mapping.
Setting F(x,y)=\langle B_{1}x,y-x\rangle and G(x,y)=\langle B_{2}x,y-x\rangle , it is easy to show that F and G satisfy conditions (A1)-(A5) as B_{i} ( i=1,2 ) is an \eta_{i} -inverse-strongly monotone mapping. Then it follows from Theorem 3.1 that the following result holds.
Figure 2.Under the global treatment, neither passenger sees Prime Time and the situation is much simpler. The first user to open the app automatically takes a ride, and the second is out of luck. Since a single ride is always taken, 1 is also the expectation. Comparing these two universes, we see that the global average treatment effect, i.e. the ground truth we would like to estimate, is given by
Subsidizing Prime Time results in a 1/3 increase in rides in our simple model. This treatment effect is of course unobservable in real life. So we must find some way to estimate it.
The standard way to A/B test an intervention like subsidized Prime Time is to pseudo-randomly assign users (in this case passengers) to either the treatment or control group, for instance by hashing their user IDs into buckets. In our example, the average result of such a randomization is that one user sees Prime Time while the other doesn’t — as illustrated by the following picture. This scenario corresponds to yet a third (mutually exclusive) parallel universe!
In order to estimate the effect of the treatment on a metric of interest for a random-user experiment like this one, one typically does the following:
Let’s see what happens when we apply this logic to our simple example. Remember that each user has a 50% chance of opening the app first. Let’s first consider user B, who happens to be in the treatment group (subsidized Prime Time). In our simple model, she is guaranteed to request and complete exactly one ride if she opens the app first. On the other hand, if she opens the app second, she will complete one ride if and only if user A decided not to request. That event also happens with a 50% probability, so that user A opens the app first, user B expects to take half a ride. Combining all this, the expected number of rides for user B is
The situation for user A is even easier. User A cannot take a ride if user B opens the app first — so the expected value is 0 in that case. And we know that user A, who sees Prime Time, will request a ride half of the time given driver availability. So the expected number of rides completed by A is
Now let’s compute an estimate of the percent change in our metric due to the Prime Time subsidy.
Obviously, this is much bigger than the ground truth effect size of 33% that we calculated above — we overestimated the effect of the Prime Time subsidy by a factor of 6! Admittedly, two users is not very many, so you might think that this fictional A/B test is suffering from small sample size problems. Surely, a user cannot actually take 0.25 rides. But imagine that the real Lyft network is composed of copies of this 2-passenger sandbox, all evolving independently over time, replenishing drivers and passengers at a constant rate. We can construct a much larger scale example, with many such boxes, where all of the above calculations still hold.
What happened in the above example is due to a statistical phenomenon known as interference (not to be confused with inference). To properly define it, we first have to introduce the notion of a potential outcome . The idea behind potential outcomes is simple: every experimental unit (e.g. user) walks around with two pieces of paper, one in each back pocket. On one of these papers is written that subject’s inevitable outcome should she happen to be assigned to the control group. On the other, her outcome given assignment to the treatment. Together, the two pieces of paper are a unit’s potential outcomes — the set of things that could potentially happen to her if she participates in the experiment. Typically, these outcomes are considered fixed and deterministic — the only thing that is random is the unit’s assignment to an experiment group.
A key assumption of causal inference is that what’s written on those two pieces of paper is unaffected by the experimental assignment that the unit happens to get, . Interference occurs when the group assignment of unit A changes any of the potential outcomes of unit B. This is precisely what we saw in the toy example above, with the outcome of interest being whether or not a ride is completed. When user A’s Prime Time is subsidized, user B is less likely to be able to complete a ride (regardless of whether or not user B’s Prime Time is also subsidized).
In medical statistics, the notion of interference arose in the study of vaccines for infectious diseases. The effectiveness of a vaccine on one subject’s outcomes depends on how many others in his social circle also received the immunization. In other words, one subject’s treatment can offer protective benefit to other, possibly untreated subjects. The result is that the measured difference between treated and untreated subjects (the benefit attributed to the vaccine) will shrink. Above, user A’s Prime Time was “protective” for user B’s propensity to successfully complete a Lyft ride — which in this case led to an of the true effect size. In general, interference bias can occur in either direction.
Lyft is not the only technology company trying to mitigate statistical interference in A/B testing. Researchers from
Lucky Brand Womens Lucky Emmie Ballet Flat Bronze 1o9QQgG
and have observed the same phenomenon in applications where advertisers or users interact within online auctions. Coarser randomization, say at the auction level, can help (but not completely) mitigate the bias. The eBay example is particularly germane to our toy example as the authors characterize interference bias in relation to supply and demand elasticity. The interference problem also occurs in experiments for social networks, where a user’s response to treatment may contaminate adjacent nodes in the graph.
Paprika Suede Womens Ballet Flats With Silver Buckle Brown o7rV7
has been made on this problem for relatively static networks, with graph clustering playing a central role. Complicating things in our world is the fact that the Lyft network is both two-sided (passengers and drivers) and has a graph structure which is incredibly dynamic. Thus interference is difficult to model explicitly.
Randomizing users is certainly not the only way to construct online experiments in a ridesharing marketplace. One can alternatively randomize app sessions, spatial units ranging from square blocks to entire cities, or even time intervals. The coarser these experimental units, the stronger the protection against interference bias in your effect estimates. However, the cost is increased variance in your estimators, because coarse units are naturally less numerous than fine units (variance scales as one over the sample size) — and sometimes just as heterogeneous. This cost can be substantial. Nevertheless, alternating time intervals between global control and global treatment configurations was a successful strategy for the Lyft Marketplace team in the early days of experimentation. The table below positions these various randomization schemes on the continuum of bias-variance tradeoffs.
To rigorously quantify these tradeoffs, however, requires a careful simulation study. Before we can embark on that adventure, we need to describe the elaborate simulation framework designed and built by the Lyft Data Science team. Luckily, that is precisely the subject of our next installment of this blog post, . Stay tuned!