quantam

The personal website of Edward Yu.

Nassim Taleb calls Nate Silver totally clueless about probability: who is right about election forecasting?

Background

Perhaps lost in the whirlwind of presidential name-calling, a lesser-known multi-year old feud has resurfaced on Twitter this election season. Nate Silver is the founder of FiveThirtyEight and is a popular statistician frequently called upon by media members to give commentary and expertise on election forecasting. Nassim Taleb is a statistician/quant turned philosopher, perhaps most well known for authoring the book “The Black Swan”. He is second most well known for calling people names on Twitter. In this 2018 instance he seemed to take issue with FiveThirtyEight’s election forecasts, saying that “klueless Nate Silver” “doesn’t know how math works”, among a host of other insults. Silver responded that Taleb was an “intellectual-yet-idiot”, an phrase coined by Taleb himself. Ouch. A litany of statisticans, mathematicians, and data scientists came out of the woodwork to take sides. Taleb himself doubled down on Oct. 10, 2020, again calling Silver “totally clueless”.

Read More

Policy Gradients from Scratch with Python

Introduction

Read More

Headless Browsing with Python 3 + MacOS Sierra

After trying for hours to find a way to automate browsing tasks with Python 3 on MacOS Sierra, I finally found the simplest way to do it. Avoid PyQt and PySide at all costs, they are a nightmare to install.

Read More

The Wisdom of Crowds

The wisdom of crowds is an extremely intuitive idea: the more people we have, the more knowledgeable that group is. It’s why democracy is based on the common vote, and why juries have more than one person. The mathematics behind the wisdom of crowds is simple and elegant.

Read More

Generalized Bayesian Updating

In Bayesian statistics the fundamental equation comes from Bayes rule, which defines how to update a posterior probability distribution given a likelihood distribution and a prior distribution.

Read More

Preventing Overfitting in Quantitative Finance

Recently, I gave an informal talk at Columbia University about how to prevent overfitting of algorithmic trading models. Personally, I think there is a general lack of statistical rigor in finance, with many traders not following recommended best practices in data science.

Read More

Life Advice From Brian

Mountain pass

Read More

The Unreasonable Effectiveness of Linear Regression

The hype around machine learning and deep learning has exploded these last few years. Every company wants to do “machine learning” now, often with a poor understanding of what that actually entails. Robin Hanson’s stance:

Read More

Why you should always guess on the SAT

Some standardized tests, like the SAT Subject Tests, have a guessing penalty. You get 1 point for every correct answer, 0 points for a blank answer, and -.25 points for a wrong answer. The test is usually in multiple choice format with 5 answer choices. The conventional wisdom that was drilled into me in high school is to leave the answer sheet blank if you cannot eliminate any of the answer choices. On the contrary, I argue that you should always guess an answer rather than leaving it blank, even if you are not able to eliminate any of the answer choices. This is because the SAT is structured such that you can take the test multiple times and submit your highest score. Thus, if you have the ability to take the test two or more times, you statistically benefit from guessing on questions you do not know.

Read More

Medical researcher re-discovers integration in 1994

I recently came across this amusing paper published in 1994.

Read More