Confidence is in the numbers - Harry Crane

Episode 4: Confidence is in the numbers with Harry Crane

Harry: At least if you're aware of these things and you're aware of some of the possibilities for these biases, you can make ad hoc types of decisions to try to maybe downweigh certain things and try to avoid overconfidence in your model.

Michael: From CMC Markets this is The Artful Trader.

Michael: Hello and welcome to The Artful Trader. I'm Michael McCarthy, chief market strategist at CMC Markets, Asia Pacific. In our third season we talk to the experts in their own fields to uncover what gives them the confidence to succeed. We uncover confidence, unlocking the secrets behind resilience, preparation and growth, and how it can make you a better trader. Today we're speaking to Harry Crane. Harry is an associate professor and chancellor's excellent scholar in statistics at Rutgers University. He focuses on probability, statistics, logic, and the foundations of complex data analysis. Harry talks to us today about both the mathematical side and the emotional side of any probabilistic situation. Revealing how intuition is just as important as hard numbers and perfect models. That's a big job title Harry.

Harry: I guess the important part of what that means I guess is, so I'm affiliated with statistics and a little bit with philosophy. But my main work is in probability and statistics, particularly in trying to understand how probability applies in complex settings. And how to use statistics to analyse data coming from complex systems. And if you go into kind of a stat 101 introductory course, a lot of the initial examples, the two examples that you're presented with somehow involve coin tossing, computing expected values, these perfect idealized models. But you know, as anyone who knows and the traders that you work with and you would know very well, that's a very small and very limited part of the big picture.

Michael: One of the areas of concern about the application of analysis and statistics has been in political polling and we've seen a number of very important votes over the last few years go against what the analysts predicted. President Trump’s election was a surprise to many people. The vote in Britain to exit the EU was a surprise to many people. What's going wrong here? Or is anything going wrong? Should we expect outlying results?

Harry: There's always going to be outlying results. Whether there's something wrong in this or not, I actually think it's probably a little bit of both. But I do think that there was something kind of fundamentally wrong in some of these analysis, the one that I'm most familiar with is in the US election, the presidential election. This is a good example for where we both get a piece of information, but it turns out that we both got that information from the same place and likely for the same reasons. You know, there were a lot of people who, when Trump was elected were in disbelief. They were actually shocked. They had no idea that this could even be possible. This was inconceivable. And you know, what's the reason for that? Well, one reason is that, you know, a lot of information coming through the news was saying, you know, was explaining every single day pretty much how you know, improbable or unlikely this was, they were giving probabilities to Clinton in the 90% to 95%, 99.9% range, making it very unlikely that Trump was going to win. And so it seemed like all this information was mounting to make it such that, well, everybody on TV is saying that this is unlikely. The statistics are saying it's unlikely, and then it happened the other way. Well, how could that possibly be? And I think that part of it is, you know, those people who were talking on T V are not independent replicates of each other. They're not all giving an independent opinion. They're all influenced by each other. They all probably come from similar backgrounds and share similar beliefs themselves. So they would have similar biases. People who only saw on their Twitter feed pro Clinton things would tend to believe that everybody in the world must be pro-Clinton. Similarly, if you were pro-Trump, you're tending to be around people who probably are pro-Trump so you don't see the other side. So I think a lot of it is a tendency to kind of see the quantity of, well all of these instances of evidence that seems to point in one direction and not realizing that these are correlated events. These are things that don't represent a lot, you know, a big signal in one direction. It represents maybe one signal that's being kind of propagated and being kind of repeated, you know, throughout this network. You know on the other hand it's important to say that there really isn't right now any kind of go to way to get around that. I mean there's no developed theory or methods to handle this complex dependency. That's part of the work that I'm trying to, you know, do, that's research that's kind of on the cutting edge. But at least if you're aware of these things and you're aware of some of the possibilities for these biases, you can make, you know, ad hoc types of decisions to try to maybe downweigh certain things and try to avoid overconfidence in your model.

Michael: So traders and people in looking at probabilistic situations, we have to know when to break the rules?

Harry: You know, the technical analysis is in a lot of ways the least important part sometimes you know, the practical side first, understanding and being able to assess and spot opportunities. You know you come across these opportunities, I'm sure it's the same in trading, you know, is this worth pursuing? Is this even worth analysing? There's a lot of situations where I'm sure after you have so much experience, you can see right off the bat, well this is obviously a good opportunity or this is obviously a bad opportunity, I'm not going to waste my time on that. And so that's the, you know that's the practical side. And then if you get to the technical side, if you have time for it, that's all well and good but then there's the you know, the whole emotional and psychological aspect of risk. Which is, well you can have the perfect model and you can have exactly the perfect strategy, but there's always going to be deviations from what you expect. And so, you know, a lot of times, you know, the first thing you talk about is whether something's a positive expected value opportunity and that's kind of the first order of thing to look for. But then of course there's the second-order effects, there's going to be volatility and there's going to be variants. And so then there's of course immediate considerations, one, can your bankroll handle those fluctuations and can you emotionally and psychologically handle those. And so those are things that don't get taught in the classroom, you know those are not academic aspects of probability, but those are very important and arguably, you know, more important I would say. And so I try to emphasize those. And I have an example of that. There's actually also ethical considerations as well, which is that when you're in the real world and you're taking risk, you're actually taking risk and that you can, if it goes one way, you win, if it goes the other way you actually lose. And so I was teaching a class once, this was actually one of the first classes I ever taught, which was at the University of Chicago. And I was doing an example, an illustration of something called the St Petersburg Paradox. I don't know if you've ever heard of that. But this is a coin-tossing example where I'm going to toss a coin until it comes up heads. So it could take me 10 tosses, could take me five tosses, but every time I toss a tail, the amount I have to pay you doubles. So we start at a dollar and then if I toss a tail, I owe you $2. If I toss another one, I owe you $4 and so on and so on. And so quickly, the stakes can go pretty high. And so I did this example in class and I actually tossed eight tails in a row. And so I ended up having to pay out the stake of the game was $1, and one of the students in the class, I guess, won $256 off of me. And so that was a real-life example, where you kind of have to deal with that volatility, but the interesting thing to me, the thing that I didn't even expect was that after we did this example, the kids in the class didn't even expect me to payout. They thought, well, this was obviously just an in class demonstration, you're not going to actually pay the person, which I did right on the spot in the classroom. So, you know, it kind of dawned on me that there's a lot that goes into this that really doesn't quite you know, can't quite be conveyed in a classroom setting. And so that's something that I've tried very hard to stay aware of those things through my outside kind of work and more practical work and more applied work. And I also try to convey those things whenever I teach them.

Michael: We've spoken with John Netto, The Protean Trader a number of times for The Artful Trader podcast and he speaks exactly to that. That he does rely on his intuition to determine his trade size. And some of his most successful trades have come when he was uncomfortable, but felt compelled to increase his position size. This is sort of going against the rules of trading. You know rule number one in trading is stay in the game, it's about protecting capital and yet, John, who's been very successful, breaks that rule regularly.

Harry: Well, I guess if it's working, you know, who am I to say that it's, you know, that it's a mistake right. But you know, I guess that's just the kind of thing, I was going to actually make a comment in the other direction, which is that, you know, mathematically, you know, there's expected value. You can talk about something being a positive plus EV, positive expected-value play, and the idea being that you should bet if it's positive and not bet otherwise. But of course, any professional gambler knows that that's not actually the case. There are other considerations and I assume, you know, gamblers have known this for hundreds of years, you know, instinctively they've known something like the Kelly criterion intuitively. You don't place big bets on small edges that are unlikely to come through, or on edges that have a very high variance. And that's kind of what the Kelly criterion formalized, right. It gave a theoretical justification for something that, you know, everybody, you know, anyone who was good or who, you know, knew what they were doing was doing already. This example of a trader going the other direction, I mean, that's kind of on another level, right. I guess that's an intuition that I guess you have to respect it for sure but as of now, yeah, I don't have an explanation for it.

Michael: Well it's possibly an insoluble conundrum, Annie Duke another guest on The Artful Trader, talked about resulting and being fooled by our results and it's an issue you've raised?

Harry: You know for sure. Just like you don't want to become kind of discouraged by a method that might be losing in the short term, you also don't want to be fooled by, you know, short term gains that are really just a result of luck. I mean, that's also part of it. I mean, the flip side of getting discouraged and having the emotional breakdown of losing is, you know, thinking that you're invincible just because you won a few times when you probably shouldn't have or that you're just, you know, running above your expected value. So that's where I think even though I think the intuitive side is in a lot of ways the most valuable. But where you're going to get your edge is how to use those tools and having a unique way of looking at it and a creative way of using them. But you know, if anything where those technical tools might be most useful is in being able to assess, you know, assess whether or not your assumptions are correct, and assess whether or not your methods are doing what you think they're doing. If after a long period of time you've been applying a method and you're running either well above your expectation or well below it, then obviously it means there's something wrong with the assumption. It doesn't mean that you should stop using the model necessarily or that it's a bad strategy. But it at least gives an indication, right, that there's something to look into and maybe a way to improve the model.

Michael: The analysis plays a sort of fundamental role in building the picture but success requires more?

Harry: Well, sure. I mean, the first thing you said was, you know, the first rule of trading was to stay in the game. And so that's something that, you know, I saw personally when I was playing poker and I had a lot of friends who played and at that time it was very popular and I was playing, you know, I was admittedly, you know, I'm playing games that I probably shouldn't. You know at that time I was playing what was the highest stakes on the internet, which was much lower than what's available now. But you have people who are very good and you know, very good at, know what they're doing, but they couldn't keep their money, right, they didn't have good bankroll management. So that's the first rule. And then, you know, so it takes a lot to be successful and it takes a lot to know what's worth your time. On any given night you can walk through a casino and find moneymaking opportunities at any given game, but you're not going to have in your head a thousand strategies for a thousand different games, when on any given night, none of those circumstances comes up. So that's where a lot of intuition comes into play as well, right. You see something, it looks good based on your experience. You have a vague intuition for how to take advantage of it. You don't have time to go and do the math or calculate anything. And based on what you know, you feel like, you know, you should either go for it or you should pass. And so that's something that I think you can probably only learn through experience. That's again, not something you could ever learn in a classroom or a textbook. And so there really is no substitute for the experience. And also, you know, you're going to have your ups and downs and losses at first.

Michael: When I first read the notes about this interview, I was surprised about your emphasis on intuition. And it's been one of the features of this series is that some of the top traders that we spoke to, Dave Floyd, John Netto, yourself, these are all people at the top of the game. Now as a young trader, the idea of trading off your gut was beaten out of me. And that was the consensus 20 years ago that we were heading towards a more scientific approach. It's clear from what you've said and what others have said that at the cutting edge of an understanding of markets at the moment, the human factor is gaining in importance.

Harry: Well, so what you just mentioned is actually something that I didn't say. You know, I talked a lot or a little bit about overconfidence, and it's easy to also have false confidence in your methods. But I didn't talk much about under confidence, which you know, in some ways is maybe not as bad because you probably won't get ruined, but you're also not going to reach kind of your full potential there in a trading or risk-taking situation. But I think that these are probably risks that are the risk of overconfidence or the risk of under confidence I mean. That probably come into play at different points in somebody's trading career right. You mentioned as a young trader, you're taught not to use your gut or your instinct because well it's expected that you don't really have any instinct I assume, this is something that takes time to develop right?

Michael: Or that they're bad.

Harry: And so I guess, how do you go about developing that and, you know, how do you have you know, the confidence at first to even go into, make your first trade or to you know even you know, step foot into this arena. At first you're probably taught some kind of system. You have some kind of process and there's people who teach you and they mentor you and so on and so forth. And there's a trust that you have in that process. And that's something that I think is also important to have. You know, it's just as important to know when to trust your gut as when not to trust it. And this is an obvious situation where if you came in day one and were overconfident and arrogant in your approach that you probably wouldn't last too long either right. But over the years as you enjoy success, then you have the possibility of overconfidence because you've been using these methods that have been proven for such a long period of time. And that's why I emphasize kind of staying sharp and trying to stay kind of aware of the many limitations. And to see different applications of a method that you might be using that actually failed or where different considerations might pop up in other applications, they're good to keep in mind as well in this whole conversation.

Michael: Harry, I don't know if you're familiar with Richard Thaler's work in the field of behavioural economics?

Harry: A little bit.

Michael: He analysed the television show Deal Or No Deal. 50 suitcases are on a board. They contain varying amounts between $1 and $50,000 and suitcases are removed. And the aim being to end up with the $50,000 suitcase. But along the way there's an offer regularly made to the player. They can take money or they can continue playing. So it's one of those rare examples of where we can see people making financial decisions under pressure and one of the key findings is that people regularly make irrational choices. I raise this issue because it's one of the challenges in analysing markets. These are complex systems with a huge number of variables, and the way each of those people reacts will not always be predictable. Does this speak to your work with complex systems at all?

Harry: Well somewhat I guess, this idea that people tend to make irrational decisions, and particularly in the Deal Or No Deal example, this is maybe a perfect example. I mean I think it's a flawed premise, it's a bad analysis for all the reasons we just mentioned, which is when you are making these decisions as a trader, you don't just see that something’s positive, expected value and then put, you know, half of your bankroll on it right. There's bankroll considerations in any of these things. So, I don't know if he's done this or not, but the first thing that I would want to think about in this case is whether somebody is making a good decision is, you know, how much money do they have? You know if you're being offered $250,000 for something that should technically maybe be worth $270,000 or something like that, you know, you're being asked to gamble something that's like $20,000 in exchange for some probabilistic event right, and that very well may be a bad gamble. If you know, maybe you're broke, you only have a few thousand dollars. So of course $250,000 is not irrational I would say. So I think there's actually a lot more to it than some of the behavioural economics literature makes it out to be. And that's, I think, because there is an emphasis in that literature and in this type of analysis on this first-order expected value type of calculation when you know anyone you know in finance or any type of risk-taking situation knows that there's second order, third order considerations about variance, skewness. And you know, what is the distribution of the payouts and then beyond just the probability distributions, of course there's the consideration of outside factors, you know, one being how much money you have, but others being, you know, how much money do you owe? You know, what are your expenditures going to be for the next month, year and so on and so forth. So all of those things are very complicated and too complicated in some way to account for all at once. But it does get at this idea that there's complexity in this decision. And that when these contestants make that decision, they're somehow, you know, in the heat of the moment and you know, in a very short and pressure-filled situation they're making those decisions. And I mean, I don't know how irrational they are by the calculations or by the analysis in these papers, but my guess would be if you actually looked at it, they're not actually that bad.

Michael: Harry, thank you very much. There was a lot of insights there for me and for our listeners, so I really appreciate you taking the time to talk to us today.

Harry: Yeah, thanks a lot. Enjoyed it.

Michael: That was Harry Crane on The Artful Trader Confidence Uncovered. If you liked what you heard in this episode, you might want to head back to episode two of Confidence Uncovered called 'Confidence In Your Worst Case Scenario' with good friends and even better traders, John Netto and Dave Floyd. I sat down with John and Dave to reveal the secrets to their success. John: It's a confidence that doesn't lead to recklessness, because confidence that leads to recklessness can be a major risk management issue. Dave: I think it's been positioned in the market as if you just do these three simple things, that's all you're going to need to be successful. But that's on top of many, many other things that are required that aren't so simple.

Michael: Make sure you subscribe on your podcast player to hear Dave Floyd and John Netto and all of our interviews or listen at theartfultraderpodcast.com. Tune in next time.