Its hard to tell whether an assessment survey has any value at all. In fact, I’m inclined to believe that most are misleading and potentially harmful.
Every survey starts with an author writing out questions, hoping that they will measure certain things. Almost always, the process ends there. Who is to know if the question actually measures what the author intended? What delineates a high score? A low score?
Our new assessment tool, currently in its last stage of development, is no typical survey. I secretly think of it as a super hero among surveys. Its not like Superman, because it wasn’t born great. Its more like the Six Million Dollar Man because it was made great through a long and expensive process called ‘survey validity and reliability assessment.’
Validity and reliability assessments answer the questions that determine if a survey has value:
- Does this measure something real?
- What are the relationships between the concepts measured?
- Are there correlations between the concepts measure and performance factors?
- What’s a high score on this survey? Low score?
You may be thinking, ‘the post title is misleading, this isn’t an update. Get to the update.’
Here it is. We’ve almost completed the entire validity and reliability assessment of our scale (lots of heavy statistical equations and computer modeling involved) and I already have the definitive answers to the bullet questions above, for you:
- Yes! our survey does measure statistically significant and meaningful concepts.
- There are many interrelations within the scale, but I can’t share the specifics right now.
- Yes! Each concept in the survey has been determined by the data to be statistically correlated to one or more performance items. I can’t share the numbers, but I can tell you that we’re excited by the strength of the correlations we’re seeing.
- We’ve figured this one out too. All data from our study is compiled in a database, against which we are benchmarking all new data. As such, the score for newly assessed companies will be translated into a percentile ranked against all existing companies in the database. In this way one can determine exactly where they stand.
In the near future we’ll be wrapping up the analysis and gearing up to present the results of the study to company participants. In fact, next Wednesday we’re giving a sneak peak of results to a company because the brass for the parent organization is in town and is interested to see the results.
I will let you know how it goes.
A new way to measure a customer or market culture
There is no easy and reliable way to measure market culture. Following the logic of management magnate Peter Drucker, there also is no easy and reliable way to manage it. But, that’s about to change.
For the past two years, we’ve been hard at work developing an organizational culture survey…one that’s more relevant and fundamental to performance than anything we’ve seen in the market…one that specifically measures market culture.
I’m happy to announce that development cycle is nearing completion and that the instrument will be unveiled for commercial use later this Spring.
Two weeks ago we completed a study with the instrument that included dozens of companies and nearly 1000 employees. We’re in the process of analyzing all study data, determining the level of validity and reliability and establishing correlations to business performance.
Because this is an exciting time for us (and should be for anyone interested in their organization’s performance) I’ll be posting periodic updates on the blog, letting you know what’s coming out of the research and how the instrument is progressing.
UPDATE: We now have a reliable and valid way to measure what many leaders call “Customer Focus” – essentially it is a culture the brings the customer into every decision point a business makes at every level. If you are interested to know more about the measurement tool CLICK HERE
Last week, in the midst of the New England Patriots run for the second ever undefeated season in the National Football League, a player from the Pittsburgh Steelers (next opponent) guaranteed victory for his team. This guarantee, made on national television, raised a few eyebrows (and fists) within the New England fan and media sets.
Today I sat in my living room and watched the Patriots methodically dismantle the Steelers, offensively and defensively. As the final minutes of the game ticked away, a chant from the home field Patriot fans could be heard to grow: “guar-an-tee, Guar-an-tee, GUAR-AN-TEE!, GUAR-AN-TEE!!, GUAR-AN-TEE!!!.” It became so loud that we watching from home could clearly hear it through the TV and feel a tingling urge to join in.
As I sat there, marveling at the perfect timing and context of this rebuke, I began to think about how sports game chants work. Stadiums fill up with tens of thousands of cheering and shouting fans. Many of these individuals are eager to start cheers and when they do, other fans listen. If they like it, they pick up the cheer; if they don’t, they ignore the cheer. The favorite cheers build momentum and burst into the thundering “GUAR-AN-TEE” that was heard today. Cheers with poor conception, bad timing, or little relevance are lost in the shuffle.
Stadiums are big cheer-factories that combine the thought-power and judgment of thousands upon thousands of fans. With this massive level of collaboration, is it any wonder that they deliver without fail?
What’s the point you might ask? The point is that if Bob Craft (Patriots owner) wanted to produce great cheers that his fans liked, his best bet would be to access their knowledge and preferences. In stadiums, this process happens automatically, but in businesses it doesn’t. If you’re interested in having the best ideas and products (cheers) at your company (stadium), take the time to listen to and collaborate with your customers (fans).