Finding the “Real” Grade for Every Climb at Dogpatch Boulders
You know the climb. It’s at a grade you normally send. You’re trying your hardest but you can’t make any progress. Is it a high-gravity day? Or might this climb be harder than you bargained for?
I used Kaya, an app where climbers can log what they’ve climbed and in how many attempts, to regrade every climb at Dogpatch Boulders according to how difficult climbers experienced it to be.
This exploration analyses 300 boulder problems, 650 climbers and more than 11,000 ascents logged between November 2022 and February 2023. I use a combination of tournament ranking and basic statistics to regrade the boulder problems.
I found that the hardest problem at a given grade assigned by Dogpatch tends to be two or three grades harder than the easiest problem at that grade.
For example, the easiest problem assigned V5 by Dogpatch is experienced on average as easier than V4. The hardest V5 is experienced by most climbers as a solid V7!
I normally project V7s at Dogpatch and have tried both these climbs. I think the new grades are accurate.
Notably, the hardest climb at every single grade is considerably harder than the average climb at the next grade.
I personally think that it’s good for problems at the same assigned grade to have some variance in their difficulty — it forces us to challenge ourselves and treat each climb as a unique experience.
Read on to learn how we rank the climbs, see lots of examples and make suggestions!
The Data Based Grades are Good But Nowhere Near Perfect
I climb a lot at Dogpatch and audited the new grades for climbs that I’ve tried or sent. I agree with around 75% of the new grades. Rankings tend to be more accurate for climbs with more ascents.
Kaya data is great but has flaws. Some climbers on Kaya always log their ascents as a flash. Climbers may put in more attempts to send enjoyable climbs while they might abandon less enjoyable climbs. Climbers who use Kaya may be different from typical climbers at Dogpatch.
Data based grades also only reflect the average challenge of each climb. If you found a climb hard, then it was hard for you and no blog post should take that away from you. I highly recommend treating grades lightly and enjoying the uniqueness of every climb.
That being said — I really enjoyed learning more about the climbs at Dogpatch and hope you do too. Review my methodology below, check out videos of key climbs and comment your thoughts!
How I Regrade the Climbs
I regrade the climbs in Dogpatch by reranking them and then extrapolating new grades.
I rerank boulder problems primarily using a “ranking tournament.” If a boulder problem doesn’t have enough data for the ranking tournament, I try to extrapolate its ranking by using the average strength of the climbers who were able to finish it.
Lastly, we set the benchmark for each grade as the median boulder at that grade and extrapolate new grades.
Anecdotally and scientifically, tournament ranking is more accurate than using average climber strength when there is sufficient data. Among other factors, some people don’t use Kaya very often and their top three climbs do not reflect how strong they really are. This contributes to the quality of our regrades — good but nowhere near perfect.
The Ranking Tournament
Climbers who use Kaya log whether they completed a boulder and how many tries they needed before sending. This gives us a lot of insight into how hard a climber perceived a climb to be.
The tournament ranking system works on the following principle: climbs that take a climber more attempts to send are harder than climbs that that climber sends in fewer attempts.
If I send Climb #1 in two attempts and flash Climb #1, I cast a single vote for Climb #1 being harder than the Climb #2. If I send Climb #3 in five attempts, I cast a vote for that climb being harder than both other climbs.
We then organize this data into a tournament.
We start by saying that every boulder problem is worth one point. In each round of the tournament, a boulder problem earns points based on the percentage of climbers who think it is harder than other climbs.
For example, if 80% of climbers thinks Climb #1 is harder than Climb #2, it will earn 80% of Climb #2’s points in each round of the tournament. If Climb #2 has a lot of points (i.e is considered hard), that means that Climb #1 will also be considered hard.
You can see an example of the tournament ranking algorithm at work below.
Ranking by Average Climber Strength
Tournament ranking is great but we can’t create a tournament entry for climbs that have only been flashed or climbs whose ascensionist hasn’t climbed anything else. This means it doesn’t work well for routes like the one below.
To fix this problem, we assign every climber a strength based on their top three climbs in the data set. We then rank every boulder based on the strength of the climbers who sent it.
In the cases where a boulder’s tournament ranking varies wildly from the average climber strength, we use average climber strength.
Let’s check out some climbs!
Unfortunately, I can’t embed videos via Substack. Check these out anyway!
All the music in these videos is by cup cup.
Let me know your ideas for further exploration!
Some ideas of my own:
Use Kaya outdoor climbing data to compare indoor climbing with outdoor climbing
Compare which climbs men experience to be difficult versus which climbs women experience to be difficult
Group and analyze climbs by style
Appendix:
Many thanks to Matt Mistele, Scott Reid and Hunter Swan for their valuable input and help.
Fun Facts About Dogpatch Boulders
Let’s play around with some of the data we have available.
The median Dogpatch Boulder climbers (who uses Kaya) projects between V4 and V5.
The most common grade set at Dogpatch Boulders is V5 (13% of all climbs), followed closely by V4 and then V6.
If you have access to movement gym's data, I am curious about the comparison across movement gym (e.g., whether movement belmont is easier than movement sunnyvale for each grade.)
Nice piece of work! You may be interested to know that there is some prior art in the scientific literature that uses very similar methodology. This approach can also be used to grade the climbers and how their ability changes through time.
2020: Estimation of Climbing Route Difficulty using Whole-History Rating https://arxiv.org/abs/2001.05388
2021: Bayesian inference of the climbing grade scale https://arxiv.org/abs/2111.08140
The second paper is my own.