Wednesday, December 12, 2012

The Ultimate Quilt Judging Algorithm



How would you feel if your art quilt was judged via a simple questionnaire rather than a panel of experts?

I read Meehl’s famous book Clinical vs statistical  Prediction: A theoretical analysis and a review of the evidence  many years ago.  I was reminded of it recently by a discussion in Kahneman’s fascinating book, Thinking, Fast and Slow, in itself a further treatise on the parlous and impossibly illogical state of human decision making!!
Meehl showed with numerous examples that in many fields a simple algorithm could make a better predictions than could experts in that particular field.  This included doctors re prognosis (remember the Apgar score they use to evaluate newborns?  It has saved many infant lives), wine-growers predicting how good a wine a particular crop will make, stock-brokers (yes! Wall street could give up tomorrow and computers calculate the best buys and sells and do it better!), financial analysts, sporting events, recidivism rates etc etc.  When I think of how much money we spend on these “fortune tellers”, instead of  on solid research and development into clean energy and so on, my mind doesn’t cogitate, it boggles!

So I started wondering if an algorithm could be developed for judging art, specifically a quilt show…or even if that would be a good thing?  We have all known of amazing quilts that weren’t accepted to shows where they should have been –  and duds that were included to everyone’s disgust (except I presume to that of the maker!!)   Would they have got in if they were assessed simply by a 6 step questionnaire?  It would also be a useful way of assessing one’s own work – which babies need help and which will be stars? I know I’m not alone in wondering which of my art works is the strongest.

Meehl concluded from his meta analyses that in order to achieve the best predictions, decisions should be based on formulae, especially in low-validity environments (like an art show).   What we also know is that the algorithm doesn’t have to include complex weighting – it doesn’t make any difference according to Dawes’ article “The Robust Beauty of Improper Linear models in decision making”.
Weighted complex combinations are no more reliable than simple ones.

 Of course “experts” are extremely hostile to these ideas, they don’t like to think that all their expertise and judgment and sensitivity counts for very little.  And they are skilled in limited, local short term situations, but longer term predictions are better assessed by a mechanical combination of a few variables. However, many have so much invested in their expertise that it makes it very difficult for them to accept their weaknesses as well as their strengths.

Okay – so which variables would we pick for judging a quilt?  Six is enough.  They should, if possible, address different aspects of the work so that there is not too much overlap.  Once the six dimensions have been chosen then a couple of questions for each one could be formulated.  For example, for me one of the important things is whether or not the piece can hold my interest – so the questions might be:

1a. How long did I look at this quilt when I first saw it?
1b. Did I come back to look at it again?

A second variable I think important would be something I’d call “freshness”.  Questions might be:
2a.  Have I seen something like this before?

And so on….so let’s see how much consensus as to important variables we would have.  So please send in your ideas!!  What characteristics of a quilt, or any work of art actually, are the most important?  Let’s see if between us  we can devise the Ultimate Quilt Judging Algorithm!

And, if you have been, thanks for reading!!  Elizabeth


17 comments:

Sandy said...

Ha, I don't claim to understand the background of your theory! But my son has recently graduated from Surrey University - Guildford. He did Physics with Finance. From what little I understand, his thesis project was based on these concepts - trying to suggest that if the financiers had used a different (more reliable?) model like the one he was proposing, we wouldn't be in this financial crisis! So, I guess the comments he got were good - especially as they said it was more like a graduate degree thesis.

and as no one seems to be wanting to offer this lad a job, he is researching it more...for his own pleasure/to keep his mind active. Maybe we should put your proposal to him?

no, he doesn't take after me but DH! well, he is also a bit artistic when he indulges that part of his brain.

Thanks for the mental gymnastics! I will be interested to see what you come up with.
Sandy in the UK

Olga said...

I find this idea most interesting. Looking at your first dimension I started wondering about the difference between attracting interest and holding interest. I agree that interest is important, but I wonder if one should be required to look deliberately for a short amount of time before being asked whether interest was held. I do believe that many good pieces can be rejected because the first glimpse is too hasty.
This proposal of yours is definitely worth taking some time to think seriously about.

Elizabeth Barton said...

good point, Olga - and of course you could have question re both - in a set of 4 or 6 quilts which one attracted your interest. Then how much time did you spend with each one...

Anonymous said...

i personally am drawn to certain colors more than others so would want to figure out a way to ignore color in order to judge the design. For me, color is always the first interest grabber -- especially red or blue, most people would have some type of bias in that area, I would think.

annabel said...

Very interesting Elizabeth and thank you for planting the seeds of an alternative way of assessing/judging. I shall cogitate thereon.

Kate Themel said...

Great article and food for thought!
In judging art (whether it's done in fiber or paint etc), I think the viewer needs at least a rudimentary knowledge of art principles. If only for the sake of being able to articulate their using common terminology.
Maybe another question, or set of questions, should be focused on fine art principles or elements of design, and how they are used in a particular piece.

I'm interested to see the final 6!

Kate Themel said...

Correction:
*Sorry, I was typing so fast I missed a word. I meant to write "articulate their OPINIONS using common terminology"

Elizabeth Barton said...

I'm going to be interested to see the final 6 too!! Nobody has suggested much though - so far!! let's hope! An interesting point raised by Anonymous - should we look at the pieces first just in grey scale??
You could show the pieces one by one just in grey scale and when the person was ready to move on they hit a key for "next"......

LC said...

My first thought was negative (such a pessimist) - Is there anything in this quilt that distresses or bothers me to the point that I wanted to 'fix' it or change it?

These are subjective, but I'm not sure anyone can come up with an objective question!!

Elizabeth Barton said...

Good one! Question could be: are there obvious unintentional awkwardnesses

Anonymous said...

If a quilt attracts my interest from a distance with color and/or design, am I still interested when viewing from a short distance?

Robin Olsen said...

Does this piece stir something in me, do I have some sort of a reaction to it? It might stir an emotional response such as compassion or anger or laughter, or it might be an aesthetic response such as awe or curiosity, but it seems any effective piece is going to have to cause a reaction in the viewer.

Nina Marie said...

This post brought to mind my system on judging science fair projects. Over the last 5 yrs I've been ask to judge a yearly 200 proj show. It seemed daunting so I came up with a number system that rated categories. Added the categories and then I could tell who I would recommend for blue ribbons. It always worked great for me but I was always amazed that the other judges saw things totally different. I was never really sure I was thinking out of the box or if they just didn't' know what they were doing - LOL - Great post - interesting!!

Anonymous said...

What about the "WOW' factor in terms of technique, which is an integral part of fiber art. Of course, we will need an algorithm within the algorithm.

Claire said...

How about "is there something about this quilt that I wonder how they did it?"

Elizabeth Barton said...

ahh some great ideas...I'll leave it a few days and then see if I can work them into a simple questionnaire...

Sally @ TCF said...

I think it is important to take into account not so much which colors were used, but if the colors used work together. I'm thinking particularly of a quilt I saw where the artist's statement said the colors were based on antique buttons. That was interesting, except the quilt was very large and the main color was .... awful (a kind of puky brown salmon.) The craftsmanship was amazing, and the quilt won a high award, but, judging by overheard comments, I was not the only person who objected to the color choice. Like all colors, this background color would certainly have worked in a different amount or situation, so it is not the color(s) used, but how they are combined that needs judging.

Might there also be room for taking into account the lack of use of technology? I don't just mean hand vs machine quilting, but photo transfer, machine embroidery, etc. I find it disturbing when those who have spent a great deal of money on machines win awards over those who have spent a great deal of time on developing less machine based techniques.