Tag Archives: SCAA

SCAA Scientific Poster presentation – The Costs of Quality

In 2010, I met Kate at the World Coffee Conference in Guatemala City. At the time, we promised to keep in touch with the view of writing a paper together, “probably about quality”. It took us 6 years, 2 PhDs, multiple trips to Costa Rica and 2 babies,  but we finally did it!

The annual SCAA Expo was in Atlanta this year, and we submitted our proposal for the scientific poster session and got approved quite quickly. So then we actually had to write the thing. Unfortunately time and money (or rather, lack of) meant I couldn’t make it to Atlanta to present it, so poor Kate had to go alone, but by the sounds of it she did it wonderfully!

Here’s the abstract we submitted:


Specialty coffee is, by definition, concerned with quality. But how that quality is determined, and by whom, is not the wholly objective process many would like to think of it as. Moreover, the details that are so important to SCAA members are often received very differently at either end of the coffee chain. Not only are many of these details misunderstood by, or irrelevant to the consumers and producers,  the focus on quality at all costs may be detrimental to both farmers and consumers.


Drawing on research at origin (in Costa Rica and Honduras) and consumption (in Northern England and Saskatchewan) as well as at previous SCAA events, we use qualitative, social scientific methods to answer a) how ideas about quality are understood and evaluated and b) how the focus on quality impacts consumers and producers.


We argue that the focus on quality to the exclusion of all other considerations is a problematic path. At origin, many farmers are not interested in meeting buyer requirements that can feel arbitrary and which seem to fluctuate daily. Others are not able to access the higher end specialty markets, and so end up selling very good coffee on the conventional market. In Costa Rica in particular the price is far too low for the efforts demanded by specialty buyers, so that many who would prefer to grow coffee have left the industry.

Consumers in Saskatchewan and northern England likewise feel excluded from, or overwhelmed by, the specialty industry’s idea of what coffee should be. In the majority of coffee shops in these areas, coffee is rarely even labelled by origin, and certainly not by SCAA cupping scale scores. Consumers are more heavily influenced by individual taste preference and by price, as well as the vendors’ marketing. With so many factors contributing to the consumers’ idea of ‘Quality’ coffee, it is difficult to determine whether  the demand for very high quality is driven by the consumers at all.

Importance and Interest:

The results of our work and of that of other social scientists indicate that the SCAA focuses only on quality at its peril. There is good reason to distinguish specialty from conventional, and of course there must be a way to indicate that distinction. To insist on only one idea of coffee is, however, to risk alienating both potential customers and current or future farmers. The SCAA does not need to embrace instant coffee to recognize that there is room to consider other factors in the determination of quality, nor are we arguing that a drive to improve growing, processing, shipping and preparation is unwarranted: rather the opposite. Indeed, the story of coffee as one of the humble, hardworking farmer who transmits love and care  through the coffee chain to the tattooed, bearded barista who then prepares your beverage with that same love and care is a compelling one. We suggest, however, that particularly in light of the coffee rust crisis and the success of ‘basic’ coffee brands using lower quality beans, there is room for more nuance and complexity in that story. This would benefit farmers who are marginalized by low prices and unrealistic expectations as well as shop owners who want their customers to love coffee as much as they do, if not in the same way.

Scientific posters are HUGE:



Here is the full thing, if I can upload it:


Leave a comment

Posted by on June 23, 2016 in Uncategorized


Tags: , , , , , ,

The Great Coffee Taste Test

… was absolutely exhausting. Which is my excuse for only getting round to writing about it now.

The Premise?

Get two coffees, one “good quality”, one less so. Get as many people as possible to try them. Find out if people preferred the high quality or not.

The Venue?

Last week, taking over what currently passes for the ‘Cafe’ on C-floor of the Geography Department, University of Sheffield.

The  partipants?

Simon from Pollards Coffee Company and I, manning the espresso machine, and over 100 willing participants from the (mainly geography) students and staff. Not exactly coffee gourmets, but certainly plenty of addicts (I quote: “I’d be on drip if I could plug it in”)

If only it were that simple!!

This experiment was not a full-proof design anyway, but a few things ‘distracted’ from the scientific precision, shall we say.


The New C floor "cafe". Ahem.

The venue served us pretty well once Simon had lugged the Fraccino espresso machine up in the lift and plugged everything in. There (as yet) is no furniture in the cafe, so there was very little for our participants to sit on, let alone write out our reply slips on whilst holding two cups of coffee…. but not the end of the world. However, although I had arranged to hold the taste test on Tuesday and Wednesday several weeks ago, it was left until Monday until someone informed us that there was a department open day on the Wednesday, who also wanted to use the space. Great! I thought. More people! Sadly, no, they all appeared at 4pm, just as we were packing up. A few rather lost looking parents-of-prospective-students appeared though, looking for the department cafe… and only found us. We were of course, happy to oblige but methinks for the sake of the geography department in general, it would have been wise to build and furnish the cafe first, and then put the signs up to it…. And I can suggest someone very willing to take over the space and run it as a cafe properly, by the way... hint hint.


In terms of research design, we had tried to keep the experiment as simple as possible. We gave our participants the sort of coffees they asked for,  cappuccinos, americanos, etc. because we wanted to test the average coffee drinker, drinking what they normal would in a coffee shop. We hoped this would give a more accurate picture of how people taste coffee – the differences should still be apparent even with added milk and sugar etc. If they are not, this is still significant because it implies that it would not matter what sort of coffee goes into a cappuccino, if people are just drinking them for the milky flavours. Also, we were likely to get far more participants this way, than if we forced them all to drink espressos. However, it does leave it open to flaws in the consistancy of our drink construction though – we may well have added more milk or foamed one better than the other or screwed up the espresso at times etc etc etc. Not a highly accurate test in this sense!


Simon setting up.

The biggest test for us though, was how to define ‘quality’ in the first place. A very large proportion of my whole thesis revolves around this issue! Simon helped a great deal here by basically using his knowledge of roasting and then absolving us of responsibility for defining quality. He chose three coffees for an espresso blend, that had all rated very highly on the Speciality Coffee Association of American’s cupping scale. Then, he got three more lots of beans from farms very close by to these first three, but had not been rated by the SCAA. Another factor was the price. The SCAA-rated blend would have cost £12 a kilo, the other, £3 a kilo. Was the ‘higher quality’ one really four times better? Pollards people roasted both sets of beans identically and on the same day, so there was as little variation in the roast as possible. All was set!

We both did this test blind – the beans arrived in bags marked A and B, and we had two grinders, also marked similarly. We then marked all the cups before giving them to our guinea-pigs.  Neither of us knew which blend was which as we made the coffees (although Simon worked it out pretty quickly!), so we couldn’t unconsciously make one coffee better than the other and so on. This is a further issue regarding quality. Quality is not just an inherent characteristic of the green bean – it also depends on the roast and the skills of the barista (amongst other things). We could just about controi these variables, so hopefully all we were testing were the difference in quality of the beans themselves. Complicated, though!


Moi as experimental barista and bemused guinea-pig


More caffeinated guinea pigs.

The Results?

Pretty evenly split!!!

This is a fascinating result and is already causing controversy at Pollards and with Simon’s suppliers. I am not going to put the exact figures on here – I am still waiting for some responses anyway, but mainly because we want to put together a proper paper about this for academic and hopefully some trade journals. For simplicity’s sake, there was no significant difference between the number of people preferring Coffee A to Coffee B. There were a very few people who couldn’t tell the difference at all, but not nearly as many of these as I had thought there might be. If anything, there was a very very tiny skew towards a preference for A, but not enough to make definite assertations.

B was the SCAA rated, high quality blend, worth four times the price of A.

I’ll let Simon explain the origins of Coffee B:

“The El Salvador is La Avila Estate, which came 5th in the 2009 Cup of Excellence awards with a score of 89.43. It is a fantastic cup on its own in a filter of cafetiere but is it worth 4 times more than the standard SHG from the farm next door?
The Brazil was a Daterra Special reserve coming in with a score of 85.5 last year. It is the Catuai varietal and again is fantastic on its own in a filter. But again is it worth the extra?
The monsoon was specially prepared for me by a neighbours estate, Ratnagiri, in Chikmagalur. It costs just the same , but, unlike me he is very good at producing exceptional quality green coffee. This does not carrry an SCAA rating but does have the indian coffee boards gold medal for last year…. “

Coffee A came from neighbouring farms in El Salvador, Brazil and India, but did not have these accolades (a fact which could be used to advocate the idea of ‘terroir coffee’ and geographical indicators… but that is a whole other chapter!).

And roughly half our participants (including me, to my surprise) preferred the cheaper, unrated Coffee A.

What does this actually mean?

Because it was such an even split, I can’t conclude that people actually prefer cheaper, supposedly lower quality coffee, because an equal number did prefer the high quality one. Judging by the comments on the day though, no-one really thought that B was worth four times as much as A.  The overriding conclusion, however, is that Quality and Preference are NOT the same thing. In short, and within reasonable parameters, (ie: not mouldy, not stale, not burnt) the quality of the beans is not a real factor in coffee preference. Take away the price (a major factor for consideration amongst students!), the marketing, the certifications and accolades on the beans, and the comfy pulls of coffee shops and their fashionable social spaces, and really, any coffee seems to be useable – and drinkable.

Where does that leave the idea of “quality” amongst coffee producers and retailers then? For the producers, all the highly skilled techniques employed to enhance the quality of the green beans are not necessarily demanded by the average consumer. However, since producers do not deal directly with consumers, if the buyers and importers are still willing to pay the farmers more for what they consider to be high quality, then it is still in the interest of the farmers to keep the quality as high as possible.

For retailers, however, this does appear to give businesses the perfect excuse to buy in cheaper, low quality coffees, and still sell them to consumers for the same price. Why would any coffee shop want to pay £12 a kilo for coffee, when half their customers are quite happy to drink stuff that costs just £3 per kilo? Quality in this case is very much constructed by the retailers: packaging, exoticism of the country of origin, certifications and labels, cafe branding, presentation of the drink (Simon is going to do another test looking at people’s perception of quality between coffees served with latte art and those without) – and also, price. If coffee costs more, the general assumption is that is must be better quality. Are we naive in this view? It is self-perpetuating – it is thought to be high quality because it costs more, and it costs more because people think it is high quality.

I was reprimanded last week for calling the SCAA’s scale, and various certifications ‘meaningless’ in my thesis chapter. As my supervisor rightly points out, the certifications and quality assurances are not in themselves meaningless, but the meanings they actually represent are not necessarily the most obvious – or what the customer believes them to be. In this case, the SCAA scale is not actually meaningless, but instead of the ratings meaning that the El Salvadorean beans are in the top 15% of all the beans tested, it means that a handful of the self-appointed experts at the SCAA liked the taste of them enough to give the coffee a high score – and in doing so, also gave retailers and importers the leave to charge four times as much for those beans than those from a neighbouring farm. However, this is NOT a quality rating; the points simply show an SCAA taste preference. If, like half of our participants, you happen to agree with the SCAA, then that coffee will be high quality to you. If you prefer the other blend, then so be it; personal preference is, after all, personal, and as our test seemed to show, “quality” is entirely subjective as well!


Posted by on April 18, 2010 in Uncategorized


Tags: , , , , , , ,