Evaluating Evaluation

It's draft season, both in the NBA and the NFL meaning evaluations of players are flying. Everyone, from general managers of the actual teams to amateur scouts on the internet are putting out their reports. Some of those amateur scouts will somehow have better results than those professionals whose very jobs it is to nail those evaluations.

How is that possible? I have no idea and that will become increasingly clear as you read the rest of this post. That is because evaluating talent is as much about who a person is as what they do on the field. Team personnel (and professional media evaluators like Mel Kiper and Todd McShay) get to talk to the draftees, their college coaches and the people around them. The amateurs on the internet? They might know a guy somewhere, but likely they're watching tape and drawing conclusions.

So how is it that the professionals can so wildly miss? Simple. It's hard to evaluate how a human being will grow from the time they are 21 or 22 (or in the NBA's case, 19) over the next ten years of their life, but knowing what to look for is half the battle.

The genesis of this post started a few weeks ago as I heard people talk about evaluating quarterbacks. Talk show hosts and the like love talking about measurables. Carson Wentz from North Dakota State has "prototypical size" and a cannon for an arm. Those are great, but when it comes to the quarterback position, the number one attribute is decision making.

I would rather have a quarterback with an average arm that is smart and decisive than one who has the physical tools and hope that the second guy can figure it out. Everything happens in the NFL at extraordinary speed. The margin for error is nearly non-existent. A quarterback must read coverage, determine where to throw the ball and deliver without hesitation. So long that he can do that, and has a baseline requisite arm to do so, he's going to succeed. The difference in arm strength between average and good isn't enough to make up for the difference in decision making between good and average.

While the decision making trait is unique to quarterbacks, there is another set of mental traits that is just as important as any other physical trait when evaluating talent. It's described perfectly by Redskins GM Scot McCloughan in a Q&A with Bleacher Report's Jason Cole.
You watch five or 10 plays, you can see the physical skills. You can see it pretty quick. If it’s a receiver running a route or a pass-rusher, you can see him drop his hips, that kind of stuff. That’s the easy part. The tough part is figuring out the person. Is he a competitor? What’s his toughness? What’s his mindset? I’ve been around long enough, which is great because I’ve made a lot of mistakes, and I’ve learned from them, but what makes a guy average to good, good to great and great to exceptional? That’s the hard part because the talent is there. Every year, you see guys come out, and they are physically gifted, bigger than life, whatever. But you also see guys who are in the sixth or seventh (round), or they are college free agents, and they play 10 years while the other guy plays two. It’s the "it" factor, and it’s hard to find. It’s really hard to find.
McCloughan's theory is true in other sports as well. Think of the great athletes and what you think of first with them. Whether is Michael Jordan, Kobe Bryant, Tom Brady, Jerry Rice or any other number of legitimate all-time greats, the first thing you hear about is their work ethic. It's how Stephen Curry elevated himself to the top of the basketball world. He had a set of natural skills that he enhanced by an immense amount of work. He wanted to be great. He loves the game. He put in the work. He is great.

These characteristics can continue to play out into a players career in a negative way as well. While Curry and the others mentioned (and many more) continually improve as they go, other players can't seem to be bothered too.

Josh Gordon led the NFL in receiving in 2013. He's played in five games since. Gordon has been suspended multiple times for failed drug tests, and while reading Gordon's accounts of his downfall creates sympathy and empathy, it also makes you take a step back and go "hey guy...figure it out!" For someone that seems to so clearly understand the circumstances around his peril, he's awful at avoiding it.

That continues to this week's news when it was revealed Gordon is reportedly living with Johnny Manziel, a walking embodiment of a similar theory. (Update: the report, from ESPN, is false according to another report from Pro Football Talk, so credit to back in Gordon's corner if it is. He still did something to have his re-instatement denied. He can appeal August 1st.)

Manziel is a master (well, until this week) of saying the right thing while doing nothing of what he says. Both of them have an optics problem and neither seems particularly interested, based on their actions, of fixing them. I should be perfectly clear here that the Manziel optics problem doesn't include his domestic violence which is a real problem that, if it played out as reported based on his ex-girlfriend's accounts, should land him in jail, nevertheless out of football.

Evaluators need to be able to identify players, or more accurately people, like Gordon and Manziel so they know not to invest vast resources in acquiring them. They also need to know that if they do acquire them with low resources (whether that be a late round pick or a low dollar contract) that they will need to invest a high number of resources to help the player succeed. The results when this is done right can be magical.

Dez Bryant dropped in the draft because he was "high-risk" coming out of Oklahoma State. Bryant has succeeded because he loves football above all else (which is a credit to him and him alone) and because the Cowboys set up a support system to make sure that he had the highest chance possible to succeed as he adjusted to a new life.

Of course the difference between Bryant, Gordon and Manziel is that only one of them has football as their highest priority. In a profession with such a short career span that requires an inordinate amount of physical dedication, that's something an evaluator has to identify.

It's why Kristaps Porzingis will succeed in the NBA and countless other European players haven't. The busts fail for the same reason many American born players have. They just don't love the game. Often these are big men who play because they're big. They can succeed without immense work because their physical stature gives them such an inherent advantage at the lower levels. In order to succeed in the world's highest level professional league, even the most gifted physical players need to have a work ethic to maximize their talent.

Porzingis loves the game. He's had early success and I'll be shocked if he doesn't continue to improve. The same is true for Karl-Anthony Towns. Last year's #1 overall pick has been glued to Kevin Garnett, who is as much coach as he is player at this point, trying to learn what's made the 15-time All-Star successful.

So as you read the latest mock draft to see who your favorite team is going to take, make sure to read the whole scouting report. Don't just look at 40-times and verticals. Take a look at the personality section. Do they love the sport? Do they value being a good teammate? Will they be a positive influence on your team's culture?

If you want to put on your amateur scouting hat, don't just watch a highlight tape. Watch how hard they play. Read and watch interviews with the player to see what kind of answers they give. What's their attention to detail?

If scouting was solely about finding the best physical specimens, it'd be pretty easy. Everyone would have the same lists. However sports are played by human beings which means there are human dynamics at play and the ability to find and project talent is an extremely valuable talent in its own right.

Inevitably, someone will cost themselves their job (and the jobs of those below them) because of ego. General manager X will think he's the one who's got it all figured out. He's the one who can take the physically gifted kid and mold all that "talent" into a great player. What the great talent evaluators have figured out is that "talent" is overrated. They want the guys who can think the game so effortlessly that they don't have to think at all, maximizing the physical ability they have leading to success.

As a fan, just hope that person isn't running your team. If they are, don't worry though. They won't be for long.