Throughout the years, I have been involved in numerous seminars on the complex and ever-changing topics of probability, prediction, and variation. Each time, I discovered that it was far more crucial to explain the intricacies of the journey rather than simply providing superficial advice on how to choose wagers or investments. Why is this? Because in a world of constantly shifting variables, static models are insufficient.
What might have been an effective system for assessing health risks or selecting winning racehorses at Santa Anita 50 years ago could be utterly useless today. The fact is, key variables are always evolving. From advancements in cancer treatment protocols to shifts in racetrack biases favoring early speed horses at certain courses, everything is subject to change. In essence, there is variation within variations themselves. This is why staying up-to-date and flexible is essential for accurate predictions and successful decision-making in an unpredictable world.
Horse racing is a sport filled with endless variation, and as a seasoned professional, I often turned to my own horse’s ever-changing performance as the perfect example during seminars. In this article, I will delve into the lifelong racing journey of one of my favorite horses, Blue Dancer. Through this personal account, I aim to demonstrate the critical elements of analysis and prediction that come into play when determining a racehorse’s likelihood of winning. From unpredictable odds to strategic planning, my experiences with Blue Dancer encapsulate the complex world of horse racing and its inherent uncertainties.
With a coat of deep, midnight blue and a sturdy but unremarkable build, Blue Dancer was not considered a great thoroughbred. But he represented the hardworking horses that graced racetracks every day across the country. While he never achieved the fame of his more illustrious counterparts, Blue Dancer still managed to place in state restricted stakes races, even as he spent most of his days competing in common claiming races.
His racing career spanned an impressive 7 years, during which he ran a total of 72 times on various tracks around the nation. And even at the age of 9, when most horses have retired, Blue was still healthy and strong, having won 8 races. When he retired from racing he came home to the farm with me and could be found grazing in our Kentucky bluegrass pastures. He continued to reign as the dominant figure among our younger horses. Until he sadly passed away a couple years ago, the glint in his dark eyes and the playful snorts served as a reminder of his once-thriving racing days.
Blue Dancer was not known for its speed on the racetrack, so don’t expect to see any triple digit ratings here. However, let’s take a look at his lifetime speed ratings (SR), along with the dates of his races and his performance finishes:
You may question why I am using a lone horse, with an unremarkable track record, as a case study for analyzing racehorse performance. But the reason is straightforward – when constructing a functional predictive model, it is essential to eliminate extraneous variables. And in the world of horse racing, there is no greater variable than the horse itself. However, do not be mistaken, this is not just a mere example. Delving into the intricacies of a single equine career may hold the key to unlocking a deeper comprehension of the ever-changing landscape of racing.
This article delves into the essential knowledge needed for true success in the unpredictable world of predictions. And it all begins with understanding the individual horse’s performance variation in this case. Self-proclaimed experts, I have discovered, offer little concrete evidence to back up their predictive claims. Their supposed proof is shrouded in secrecy, or they rely on nothing more than anecdotes from isolated races to support widely accepted but debunked beliefs. But through thorough longitudinal studies, I have uncovered the myths that these charlatans perpetuate for their own fame.
Through the meticulous analysis of individual career data, I embarked on a series of “experiments”. These experiments allowed me to delve deep into the realm of predictive factors and gain insight into what truly drives future events. With each trial, patterns began to emerge and clear correlations between certain variables and outcomes were established.
The importance of each factor was scrutinized – were finish positions more telling than speed ratings? Did jockey changes hold significant weight in predicting results? And what about seemingly trivial details such as post position or the horse’s last race? Through these experiments, previously unknown information came to light, revealing the crucial role that certain aspects play in determining future success. It was a journey of discovery through the world of data, uncovering valuable insights and expanding our understanding of how it all comes together in the world of horse racing.
Unpredictability lurks in the shadows of even the most seemingly reliable things. No matter how much we try to control and predict outcomes, variation always sneaks its way in.
In the world of basketball, free throws are a prime example of variation. The variables remain constant – ball, rim, distance, height – yet our performance is never consistent. We can all make a free throw, but why can’t we make it every time? It’s a maddening concept that even the best free throw shooting professional players struggle with, only making about 9 out of 10 shots on average. These elite athletes, who are paid to master their craft, still cannot escape the clutches of unpredictability. It taunts them, challenges them, reminding them that perfection is just out of reach.
In the unforgiving world of machines and technology, even the most perfectly crafted car can falter in its first 10,000 miles. With each precise component built to withstand incredible performance tolerances, any failure seems inconceivable, yet it still happens. And what about those machines that seem to defy logic and run flawlessly for decades? But don’t be fooled by their longevity – they too are susceptible to inexplicable failures, as if mocking our attempts at control and understanding.
For every system and machine is plagued by the inevitability of errors and variation, no matter how diligently we try to minimize them. In this battle against chaos, perfection may never be achieved, only fleeting moments of success before the next unexpected failure strikes.
As my mind races with the exhilaration of exploring the vast sea of variation, I will force myself to refocus on Blue Dancer. My thoughts turn to the familiar frustration of missed free throws as I study the graph depicting his lifetime finishes over time. Each data point a bittersweet reminder of past victories and defeats, fueling a fiery determination to push beyond limitations and reach for greatness.
As you gaze at the chart, you may come to the conclusion that Blue Dancer’s performance results were utterly unpredictable. However, those who claim to be experts in handicapping would scoff at such a notion and instead see evidence of foolish beliefs. They lack an understanding of statistical variation and erroneously attribute false trends, like the concept of “bounces”, to explain these results.
In the world of horse handicapping, there are many disproven myths that continue to circulate. One such myth is the concept of a “bounce.” This term is used when a horse has an exceptional performance and some experts foolishly predict that their next performance will be subpar. In reality, this is simply a result of normal variation, much like flipping heads on a coin after flipping tails. It would be like saying Babe Ruth “bounced” in 1928 after hitting 60 home runs in 1927.
Unfortunately, numerous experts in horse racing handicapping have gained fame for reasons completely unrelated to their understanding of probability and statistics. Being a former jockey does not automatically make someone a proficient handicapper, just like being a former NFL lineman does not give someone the ability to accurately predict future outcomes, unless they further their education and study statistical science! But setting aside these fraudulent media personalities, let’s focus on Blue Dancer and move past these antiquated beliefs.
The past performances of Blue Dancer in his racing career were intriguing, reflecting a wide range of outcomes. He had competed at various levels throughout his career, from bottom-level claiming races to high-stakes events. Therefore, a first or fifth place finish did not always carry the same weight. To gain a better understanding of this variation and its implications for predictive data, let us examine Blue Dancer’s lifetime Speed Ratings. These numbers encapsulate the speed and performance of each race, providing insight into his overall success on the track.
While speed ratings are not flawless, nothing truly is. They may offer a directional perspective and have significant value if used correctly as a baseline for future predictions. Many experienced trainers, veteran handicappers, and racing experts may be familiar with them, but they often fail to fully grasp their potential. When utilized properly by profit-driven handicappers, speed ratings can save valuable time. Essentially, they are designed to make data easier to interpret. Unfortunately, these technical terms may be unfamiliar to traditional handicappers, similar to how Quantum Theory would be to cavemen. Nonetheless, speed rating services use algorithms to standardize data and produce comparative ratings.
Speed ratings are a crucial tool for horse racing enthusiasts, allowing them to objectively compare every race and every horse, regardless of track or conditions. However, the normalization process can vary between services, taking into account factors such as track biases, wind effects, and even the impact of running lanes on the racetrack’s turn semi-circles. To demonstrate this, here is a visual representation of Blue Dancer’s lifetime speed ratings as calculated by one particular service. A rainbow of colors cascades down the graph, each representing a different race and its corresponding speed rating. The fluctuations in speed are clearly visible, a testament to the ever-changing conditions on the racetrack.
As I reminisce about my younger days in the 70’s, I can’t help but feel a sense of conflict when it comes to speed ratings. Back then, they were my obsession, spending countless hours in libraries scouring through microfiche in order to develop my own ratings for Chicago racetracks. I worked at Kemper Insurance in those early days, skipping out often to libraries or racetracks. Now, speed ratings are everywhere and easily accessible. Part of me misses the thrill of the hunt, while another part of me is grateful for the convenience.
Upon observing the chart, one can note that Blue Dancer’s speed ratings fluctuated from race to race. Here are a few basic observations on the data:
1. Blue Dancer’s early racing career was marked by numerous races as a two-year old, but once he surpassed this stage, his speed ratings never dipped below 69. It seemed as though he had truly found his footing on the track.
2. During his prime, Blue Dancer’s speed ratings consistently stayed above 75 and even reached a peak of 92. His performances were like a symphony, each one building upon the last in an impressive display of skill and grace.
3. Towards the end of his racing career, Blue Dancer’s speed ratings started to decline, averaging between 64 and 83 in his last 13 races. It was clear that age was starting to catch up with him, but he still gave it his all on the track.
4. Despite the overall trend of a 20 point swing in Blue Dancer’s speed rating performance, three distinct time frames could be observed. The first was when he was a young and inexperienced runner, the second was when he reached his full potential as a mature athlete, and the third was when age began to impact his abilities on the track. Each phase had its own unique characteristics and contributed to Blue Dancer’s reputation as a solid mid tier performing racehorse.
Every athlete’s performance is constantly scrutinized and evaluated, just like Peyton Manning’s quarterback ratings. But for me, it’s not enough to simply look at the surface level data. I have spent my entire life digging deeper and separating the relevant from the irrelevant.
With every piece of information, I carefully analyze and question in order to establish accurate “probabilities” of potential outcomes. In a world where nothing is certain, probabilities are my lifeline, guiding me through the chaos and uncertainty of predicting the future.
The world of speed rating handicappers is filled with endless theories and angles, all in an attempt to predict the future finishes of races. Some place great importance on the last recorded speed rating, believing it to be the ultimate indicator of a horse’s performance. Others focus on the last speed rating specifically on the type of track being raced on, whether it be turf, dirt, or off-track conditions.
There are those who believe that taking an average of the last 3 speed ratings provides a more accurate prediction. And then there are those who rely on straight line trends, using recent speed ratings as a guide for potential outcomes. The air buzzes with excitement as each handicapper puts their strategy to the test, hoping to come out on top at the finish line.
As I searched for a way to accurately predict racing probabilities, I delved into various forms of data. From analyzing individual career statistics, I discovered that normalized speed ratings were the key to unlocking my predictions. With this method, I could clearly see if past speed ratings were indicative of future performance, and much to my surprise, they were. However, the resulting predictive values often defied my expectations.
As I delved into my data experiments, I began by closely examining individual speed rating data points. My goal was to understand the degree of variability between each speed rating throughout a horse’s entire career. It may sound like a straightforward task, but it required careful analysis and attention to detail. To demonstrate this process, I have elected to showcase the last 10 races of Blue Dancer’s career. By doing so, I hope to provide a clear understanding of the methodology behind my research.
When trying to predict a race, many handicappers believe that the most important factor is the previous race. In this analysis, we examine whether the last speed rating can accurately forecast the next one. Specifically, we ask if the October 18, 2008 Blue Dancer speed rating was a good predictor of the November 14, 2008 Blue Dancer speed rating. The answer to this specific question is yes, as there was only a one-point difference between the ratings (57 on October 18 compared to 58 on November 14).
Naturally, I extended this comparative analysis to all 72 of Blue Dancer’s races. This produced a total of 71 comparative data points, and as anticipated, each race showed a different level of discrepancy. For instance, the speed rating result on October 18, 2008 was 57, which was 12 points lower than the September 1, 2008 result of 69, as shown. Upon completion, my comprehensive comparative analysis throughout Blue Dancer’s career determined that the average margin of difference was 6.41 speed rating points, with some races varying more or less from this average.
My hope is that this basic illustration serves to clarify the concepts I discussed in other blog posts. The world, especially the realm of horse racing, is inundated with an abundance of information and data. Unfortunately, only a small fraction of individuals make the effort to uncover genuine truths from it all.
As I delved deeper into my analysis, my next inquiry was whether the previous speed rating before the last race held any more predictive value than any other speed rating in a horse’s career record. In simpler terms, I looked at the two races prior and compared them to each other, as shown below:
Once again, the degree of variation was carefully measured at each point, and as expected, there was a greater amount of variability. The average difference between speeds increased to 7.20 points, indicating that the second race prior was not as reliable in predicting future performance compared to the one immediately before it. This is to be expected, isn’t it?
Experienced handicappers would point out the obvious, “We all know that older races are less reliable indicators than recent ones, you idiot!” And they would be correct. However, the key difference between us is that I have access to data and the ability to experiment. I am not seeking definitive answers, but rather probabilities. So, I posed the next question: How does the third race before this one compare? And how do all the races leading up to it compare as well?
In the early days, before spreadsheets and advanced programming capabilities, I toiled away with slide rules, clunky adding machines, cumbersome punch cards, and basic calculators in attempts to solve similar seemingly trivial inquiries. But now, thanks to modern technology, these same questions can be answered within mere seconds. The passage of time has transformed the process from a laborious and tedious task to a swift and efficient feat.
The chart below shows how each of Blue Dancer’s speed ratings in his 72 race history compares to his previous 8 race speed ratings.
| Comparison Criteria | Variation |
| Prior Race | 6.41 |
| 2nd Race Prior | 7.20 |
| 3rd Race Prior | 6.93 |
| 4th Race Prior | 6.60 |
| 5th Race Prior | 6.44 |
| 6th Race Prior | 7.00 |
| 7th Race Prior | 7.21 |
| 8th Race Prior | 7.40 |
In terms of predicting the upcoming race, Blue Dancer’s performance in the race just before is the most reliable indicator. However, it’s worth noting that the 5th race prior also has a high degree of predictability. The range of variation in this chart is relatively narrow, as all eight previous races have been within a 6.4 to 7.4 point difference in predicting future speed ratings. This means that each data point is equally valuable (or unhelpful) in forecasting the outcome.
Since these results were only slightly useful, I began examining other groupings, specifically the “averages” of speed ratings. Could this have a stronger predictive value compared to just looking at a single data point? I was aware of a 6.41 degree variation in the last race prior to the one we’re trying to predict, but could grouping together several speed rating data points result in less variability?
In order to analyze the averages, I began by calculating the average of the two previous speed ratings.
For this illustrated example, I took the average speed ratings from October 18th and September 1st and got a rating of 63. Then, I compared that to the November 14th race, where the speed rating was 56. This resulted in a variation of 7.0 speed rating points between the actual result and the average of 63. I repeated this calculation for every race in Blue Dancer’s history
After analyzing Blue Dancer’s racing history, I discovered that taking the average of the two previous races yielded a variation of 6.21 degrees in predicting the next race. This means that using this method would result in a more accurate prediction compared to relying on any single speed rating. In fact, by averaging the prior two speed ratings, my predictive ability improved by 3.1%. How was this possible? It turns out that taking into account more data resulted in a decrease in absolute variation, from 6.41 to 6.21. This drop of .969 translates to a significant improvement of 3.1% in predictive accuracy.
The meticulous analysis, painstakingly expanded and conducted on a grand scale with vast samples of horses, revealed the folly of many handicappers’ fascination with the final data point. It was clear that this single point from the last race held little weight in predicting future outcomes. Just like any other data point in the world, it too was subject to the universal laws of variation. The intricate patterns and fluctuations of the racing world were no exception.
Eager to improve the predictability of my data, I took another step forward. My eyes scanned the numbers and patterns on my screen, searching for a way to enhance my predictions. Would incorporating the average of the prior three race speed ratings make an even bigger impact on forecasting the future? The possibilities sparked excitement in my mind as I delved deeper into my calculations and analysis. This could be the key to unlocking new levels of accuracy and insight in my research. So, I asked another question.
Once more, I conducted my analysis and found that the preceding three race average consistently predicted the future with greater accuracy than any individual data point or the average of the last two races. The variation in predictions was reduced to a mere 5.55 speed rating points, a significant improvement of 13.3% compared to solely relying on the last race for future predictions.
In my blog post “The One-Eyed King,” I discuss how I discovered the need to predict horse races 26.3% better than the average bettor in order to break even. This is due to various expenses and fees taken from betting pools by the racetrack. Now, let’s say the average bettor uses the last speed rating to make their picks. Well, I’ve just figured out a way to gain 13.3% of that necessary 26.3%!
Foolishness runs rampant among average bettors, who throw caution and logic to the wind as they place their wagers. With no regard for the last speed rating or any other data, they blindly hope for a stroke of luck. But for those who are truly dedicated to winning, it’s crucial to constantly experiment with data. It’s about finding every single percentage point you can, combining them, and gaining a powerful mathematical edge over your opponents in the cutthroat world of pari-mutuel betting.
Through persistent experimentation with my data, I was able to create a comprehensive table that goes all the way back to averaging the last 8 races.
| Comparison Criteria | Variation |
| Prior Race | 6.40 |
| Prior 2R Avg | 6.21 |
| Prior 3R Avg | 5.55 |
| Prior 4R Avg | 5.25 |
| Prior 5R Avg | 5.18 |
| Prior 6R Avg | 5.07 |
| Prior 7R Avg | 5.19 |
| Prior 8R Avg | 5.19 |
Upon analyzing the data, it becomes clear that the level of predictability rises with each additional recent race that is averaged. However, this trend eventually flattens out after six races. Surprisingly, the predictive value begins to decrease after considering more than six previous races. This reveals the importance of recency in making accurate predictions, and it confirms a lesson I’ve learned in the past: there is a specific point where the balance of recent and old data is optimal, and that is within the last six races – not one more or one less.
After analyzing the average speed rating of the last six races, I noticed a difference of 5.18 points in terms of predictive accuracy. This equated to a 20.8% increase in accuracy compared to solely relying on the last speed rating. These results aligned with what I had previously observed in other areas of study: a single data point holds some significance, but it pales in comparison to the insights gained from analyzing multiple data points.
According to the most basic principles of predictive handicapping, a horse’s performance in the previous six races holds more weight in determining its future results than any other subset of races. This is a crucial concept for value handicappers who are focused on steadily increasing their bankroll over time. By understanding this “sweet spot,” they can gain an advantage over other handicappers who are fixated on false beliefs and only give importance to a horse’s most recent race performance.
Throughout the vast expanse of human history, people have been consumed by the allure of recent data points. It is a familiar trend, seen in the daily newspapers and online articles that flood our screens each morning. No matter the subject – whether it be sports, weather, disease, entertainment, or the stock market – these simple numbers and statistics are easy to grasp and talk about, making them prime contenders for predicting the future.
It must be noted that the media is not driven by a desire for uncovering truth. Rather, they are fueled by ratings and profits. As a result, traditional ethics of journalism are often warped or disregarded entirely as the media sensationalizes and distorts stories based on these fleeting data points. The truth becomes secondary to generating drama, attention and revenue, creating a cycle of endlessly ridiculous and exaggerated news stories.
The constant bombardment of nonsensical data and exaggerated stories has indoctrinated the public into unquestioningly believing every new piece of information as absolute fact. Those who are knowledgeable in the ways of diversity take advantage of their ignorance and naivety, using every chance to outsmart and outperform the unsuspecting majority. In the dog-eat-dog realm of horse racing, this is evident in the overwhelming odds stacked against us, a clear reflection of the gullibility and simplicity of the masses.
A universal advantage awaits those who possess a deep understanding of variation in all forms of endeavor. Like the keen eye of a seasoned gambler, I have analyzed data and discovered that the last 6 Thoroughbred horse races hold more predictive power for future outcomes than the last 1, 3, 5 or 10. This same powerful process applies to all aspects of life, unlocking secrets and opportunities for those with the knowledge and insight to harness it.
This was just one simple data experiment. I wanted to illustrate the detail of such a simple experiment to demonstrate how unanswerable problems can become clearer. Hidden probabilities and truths often emerge in stages.
Will this past 6 race knowledge hold forever? Doubt lingers like a shadow, shrouding the answer in uncertainty. Will the percentages I reflected in here hold forever? To some degree but with each passing moment they crumble and shift like shifting sands in a desert storm.
Did I discover that this past 6 race recency probability value could be improved upon by looking at dates of the races, types of racing surface, conditions of the track, utilizing an exponential (not linear) trend calculation of speed ratings rather than simple averages to predict future results? Yes, they did.
To unlock secrets, one must dive deep into the endless abyss of data, navigating treacherous waters and braving the unknown depths. And even then, there are no guarantees, for fate is fickle and can change in an instant. This knowledge is a double-edged sword that must be wielded with caution and skill.
