Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It would be cool if you could divert only a fraction of traffic into a particular test. A future feature, perhaps?


You actually already can do that, and all of the clients support it. It's probably just not documented.

Passing ?traffic_dist=10 to the participate endpoint will only direct 10% of traffic to the test. The rest will get the control alternative.

I'll add this to the documentation.


That's not going to be the best API over time.

Give everything a default traffic weight of, say, 10. Make it alterable as above. Divide traffic proportionately between all versions according to their weight.

The reason to do it this way is so that when you go from 5 versions to 4, then 4 to 3, then 3 to 2 you just eliminate versions and don't have to calculate the percentages to rebalance. Trust me, calculating percentages gets very old, very fast.

While you're at it, add the ability for versions to be marked as not to be reported on. This allows people to have versions test, control, and unreported control. You start with a lot in unreported control. You ramp up the test by dropping the weight of unreported control.


+1 on the unreported control. It's a common use case to start an A/B test at a small test percentage, then ramp up over time. Without an unreported control, you can't do that and have valid samples for each group.

(E.g. your sample mix will be invalid if you start at 10 test / 90 control then ramp up to 30/70. But if you start 10 test / 10 control / 80 unreported, then ramp to 30/30/40, your samples will be valid.)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: