A crown had gathered, eager to hear the results of the survey.
Introduction:
where did this come from?
This inquiry sprang from a misguided notion I’ve
been carrying around since I was a teenager; that the recognition of games via
awards translated to an increase in interest – and by extension, sales – in
games generally. Around the time I began to get into wargaming, in my early
teens – this was the late seventies and early eighties – the Academy Awards for
Motion Picture Arts and Sciences were a big thing, or at least taken more
seriously than they seem to be nowadays. A film that won the Best Picture Oscar
would see a huge surge in cinema attendance as more people went to the cinema
to see it, or to see it again. Sometimes it would even warrant a theatre
re-release so all the people who missed it the first time could go see what the
fuss was about.
You see the same dynamics in publishing. Arundhati
Roy’s debut novel, The God of Small Things, was slated for a paperback
print run of around 5,000 copies after a hardcover release of about a thousand.
When The God of Small Things was shortlisted
for the prestigious Booker Prize, the publisher upped the initial paperback print
run to 160,000 copies on the strength of the nomination.
When I read about awards like the Origins Awards in
wargame magazines, I naturally conflated this recognition with greater
commercial success; it developed into an article of faith and for a long time I
never stopped to question it. Maybe back then, in those heady days before the gift of the Internet – and its pharmakon, Internet anonymity – things did work out
this way.
For me the moment of questioning came half-way
through writing a post about what I thought was the true value proposition for publishers
receiving a nomination for an award with reasonably high visibility in an
admittedly small demographic, in reply to an actual publisher venting that
their nomination didn’t translate to any real value at all. It was a profound
example of Dunning-Krueger exposition, and I considered just deleting the post
as I thought about it (it’s still embarrassing, but in the interests of
intellectual honesty, it remains up). In short, this inquiry was born out of
personal interest; I set out to find out what publishers’ experience was with
awards to satisfy my own curiosity, and I’ve written up what I found to justify
the efforts of everyone involved.
Scope and methodology
I’m not a trained statistician, but I’m something of
a data wonk and have picked up some basics in various work roles over time. I
wanted to apply some intellectual rigour to the project, and I wanted to write
it up, if only out of respect for the respondents (and, of course, for more and
interesting blog content).
For the survey to produce results I could call
representative, I needed a largish pool of potential participants. These would
be companies which released wargames either exclusively or in a sufficiently
high number of titles (50-80% of total output) The obvious choice was
recipients of Charles S. Roberts nominations. Over the last five years of the
awards, the categories have been pretty stable, so I kept my query pool to
recipients from those five award rounds. I looked only at the categories awarded
to games, not game designers or illustrators. I also left out nominees for the
Science Fiction or Fantasy category, as the companies represented for this
award were either more mainstream companies who happened to stumble into a
wargame nomination, or had already been represented in other categories.
I identified 61 publishers from the historical game-awarded
categories. I already had a conversant relationship with a couple of the
publishers on the list, so I contacted them first; the others I essentially
cold-called (via email, or in a couple of cases, a direct message via
Facebook), with a form letter comprised of four questions and an opportunity to
provide further comments. I spent about three weeks contacting – and in a few
cases corresponding with – publishers, but the essential message was consistent
for all. It read as follows:
Wargame
awards value-proposition survey
[Greeting],
I post a small blog
about wargames and wargaming, and I want to write a piece about the perceived
and actual effects that award nominations or wins have on sales of the
nominated titles, that an award (or nomination) automatically translates to
increased interest and/or sales, or a general increase in brand recognition.
I’m hoping to get a clearer picture of this with your help. I'm writing to
every publisher that has been nominated for a Charles S. Roberts Award in the
last five years, along with some others I'm aware of receiving nominations for
other awards.
I’m writing to
wargame publishers in the hopes that enough will respond to make a useful
sample group, asking a few general questions about their experience regarding
awards. I’m not asking for hard figures or percentages, or information that
might be considered commercial in confidence; only general responses to the
following questions based on your company’s experience.
I have prepared four
questions (three only requiring single-word answers) and an invitation to add
any further comments. The answers to questions 2, 3 and 4 will be used for a
simple statistical analysis. Please feel free to be as general or specific in your
comments as you wish, and you’re welcome to add a statement or any further
comments (anonymity will be assumed unless permission is explicitly granted).
1. As a wargame
publisher, what do you see as the value proposition for awards like the Charles
S. Roberts Awards or the SDHistCon Summit Award?
2. Have you seen an increase in interest or
orders of titles that have received a nomination or nominations, but failed to
win an award (ball-park only – negligible / noticeable / profound)?
3. Have you seen an
increase in interest or orders of titles that have won an award? If so, would
you consider this a significant increase (as above)?
4. Would you class
your company as small, medium or large (in terms of wargame publishers, not
compared to IBM)?
5. Do you wish to add
further comments (if so, please indicate if these may be attributed)?
Thank you for reading
this far. I will inform the survey participants ahead of the post being
published and will provide a pre-publication draft on request.
Regards,
Jonathon Dyer
https://fastgame-goodgame.blogspot.com/
-------
I intentionally kept the questions simple and
straight-forward. I figured anyone who chose reply was taking time away from
something more important to do so, and so I kept three of the questions to single
word answers, leaving open the opportunity for the respondents to expand their
answers as they saw fit.
Of the sixty-one companies I attempted to contact, one
had since gone out of business. Two more proved impossible to reach (due to
technical issues with their website-based messaging apps – I did try several
times in each case, but without success). I believe I likely managed to contact
a human at each of the other fifty-seven companies from the complete list.
Response
From my admittedly limited experience with surveys
from working in libraries, I hoped for around a 12-15% response rate, but I was
sure I’d be lucky to get much more than 6-7%; cold-call surveys rarely get
above this level of participation. If I didn’t make 5% (four or five results),
I had a contingency plan of expanding the pool to European award nominees. I
bullied a couple of publishers with whom I already had some contact into helping
out, but most were formal approaches via email. Or in a couple of cases, via
Facebook Messenger.
As mentioned, I couldn’t reach everyone on the list,
but I have worked on the assumption that I managed to reach all fifty-seven of
the other publishers over the course of about two weeks. I’m pleased – and more
than a little surprised – to report that twenty-four publishers responded to
the survey, a response rate of roughly 42% of those contacted. This was a much
better return than I had any cause to hope for, and I’m grateful to everyone
who took the time to respond.
The geographical breakdown of the respondents shows a truly international response to the questionnaire, which is heartening. By nationality, the biggest non-US response was from Italy, with France and the United Kingdom tied for second place.
Some respondents answered perfunctorily, which is
fine; I really didn’t expect any more than that (the opportunity for comments
was optional), and their contribution is appreciated. Some added comments under
request of anonymity, or of not having their comments presented at all, which I
respect; this really is a small community, after all. A handful of respondents
were happy to be quoted, and each of these made insightful comments; my
especial gratitude goes to them.
Results
What follows is an overall analysis at all of the
response material received with little extrapolation. I have tried to keep this
reading as objective as I can. The material is presented in a different order
to how the questions were presented in the questionnaire.
I wanted to look at the self-reported size of the
respondents first, because I think this has had an influence on the subsequent data
collected. The data collected from the remaining questions are presented in the
order as appeared in the questionnaire. The responses to the value, or
perceived value, of wargame awards generally from the publishers’ point of view
have been distilled into shared themes. Every publisher has their own distinct
take on the value of awards, but these are often more similarities than differences
at play.
Questions 2 and 3 are inextricably linked; there are
nearly an order of magnitude more award nominees than award winners, but I
thought it would be worthwhile to see if a game taking a gong would have any
further effect that just a nomination. Only ten of the twenty-four responding
publishers had won an award of any stripe.
I’ve tried to avoid reading too much into the data
beyond neutral observations, and where possible I have let the respondents
speak for themselves. I will not be releasing the names of the respondents a
list of the respondents; several requested anonymity, and added to this, it
would be easy enough to reverse-engineer a list of companies that, for whatever
reason, chose not to respond to the survey. I have liberally quoted from the
companies that agreed to go on the record to add some nuance to the findings.
Company size
“I’d class us as a small publisher, and I’d guess that’s the case
for most of the wargame industry, with just a few exceptions. It’s a niche
market, and even the more well-known names tend to operate on a relatively
modest scale compared to broader segments of the gaming world.”
- David Heath, Lock ‘N Load Publishing
I guess we're medium. Small would be 1-2 people. Counting all the
freelancers' time, we must be around 2-3 full time jobs for wargames.
- Florent Coupeau, NUTS! Publishing
Small – we fit in a garage. 8-)
- Steve Rawling, LPS Inc. (Against the Odds magazine)
The wargame industry is small, even in the context
of the broader hobby gaming sector it’s a thin slice from the pie-chart of
overall game production. I think an argument could be made that there are three
or maybe four “large” companies operating significantly in the hex-and-counter
gaming space located in the United States, arguably another one or two in
Europe – though that might require some renegotiation of what constitutes a
wargame, which is definitely not what I’m here to discuss – and at least two more
in Asia. None of these are large
companies in comparison to other industries, or even companies in the broader
game publication sphere. The publishers we think of as industry flagships are
still tiny compared to the likes of Hasbro, Asmodee, or Games Workshop.
The question of company size was the last question
on the survey, but I wanted to address it first because it has some bearing on
the responses to the other questions. I made the question simple to answer with
just three single-word options, but I didn’t want to dictate guidelines for
what should constitute a small, medium of large company; I was interested to
see. A small company might release a new game every other year, or they might
produce one or two a month, along with reprints of previous games. I was more
interested in learning how publishers see themselves within the ecology of
wargame publishing.
Respondents were encouraged to use a peer-comparison in identifying their size, measuring themselves against other wargame publishers inside of the ecosystem. I think inside the industry there is some cache in representing yourself as a small publisher, like a band that doesn’t want to appear that they have “sold out.” That’s not meant to be any kind of indictment; it’s merely an observation, though it has some baring on some of the data. It would be interesting to dig into more deeply in a future study, but it’s not what we’re concerned with here.
NES releases one game every 6 to 8 years, but recently that has increased
to one every 3 years. However, every game that has been released has gotten
nominated for something in the CSR Awards.
- Mark Hinkle, New England Simulations
Nearly all the respondents declared theirs to be a
small company. Most of these were one- or two-person operations, but the count
included at least two companies that, if asked, I would have positioned as
medium. Uwe Walentin described Sound of Drums as, “Small, but on the verge to
medium.”
Three publishers defined themselves as medium-sized
operations, though their reasons varied. One publisher placed their company in
the medium bracket on the basis of their larger-than-average initial print runs,
in spite of fielding a very small staff. One respondent wrote, “In wargame
terms, you are either [one company was mentioned here], more than one employee
(medium) or one employee or fraction (small).” Only one anonymous respondent
described themselves as a “large” publisher, “in the top 2-3 in the USA if not
the world.”
The value
propositon
A general once asked Napoleon what was the purpose of awarding petty
baubles like the Legion of Honor to the troops and Napoleon replied, “It is
with baubles that men are led.” The principle is true today, the awards give
everyone involved something to shoot for and in a world of inadequate monetary
compensation for oftimes heroic efforts, a bit of psychic satisfaction for the
winners.
“Regarding awards […] in general, the value proposition is like that of a
medal of honor: it may give you or your company some gratifications from single
persons or associations, but it doesn't help you to fight better.”
- Nicola Contardi, Europa Simulazioni
As a wargame publisher, it is a very useful catchphrase on the front or
back of a box.
- Yasushi Nakaguro, Bonsai Games
The first question on the list was the most
open-ended because I wanted to let the respondents answer as freely as they
felt comfortable. I didn’t expect many respondents to go into too much detail
with this one, but I wanted to get a measure of publishers’ expectations of
what wargame awards mean to their businesses. This freedom resulted in several
respondents requesting anonymity. Here I’ll present a synthesis of the results
to highlight the themes shared among the responses. I’ll also include quotes from
some respondents (with their permission) to add context.
Wargame awards are generally well-regarded by wargame publishers. Some of the respondents suggested there may be better ways to govern the nomination process or some other aspect of various awards (you can’t please all of the punters all of the time), but nearly everyone saw something positive in having wargame awards, if only for others.
The majority of respondents expressed a generally
positive view of wargame awards. Three quarters of respondents expressed a positive
opinion of wargame awards in general or the Charles S. Roberts Awards in
particular. This doesn’t equate to a tangible benefit to the publisher in all
cases. Of those who expressed a positive view, 16.6% emphasised the recognition
of their designers rather than any value for the company. Another 54.2% of
respondents saw award recognition as a positive in terms of marketing, either
passively – game or brand awareness-raising – or as a chance to capitalise on
that recognition boost through their own marketing (respondents were split
roughly fifty/fifty, with a couple highlighting both).
It's a great way to highlight good wargames to the public. It is so hard to break through these days so every new opportunity is very valuable.
- Petter Schanke Olsen, Tompet Games
“The value proposition is for industry recognition and establishing brand
awareness for excellence. While, as I
will note below, we do not see significant uptick in sales of the title, we do
believe it makes customers more likely to buy or back our games in the future.”
- Kevin Bertram, Fort Circle Games
“Awards and nominations have had no discernible impact on our sales or
notoriety. I think this is more a result of our size/niche; I'm sure other
publishers see an impact. But it's nice to have the work recognized by one's
peers and public, especially because we are off in our own corner.”
- Amabel Holland, Hollanspiele
Fully a quarter of respondents took a less
charitable view of awards, in terms of benefits. This presented more as
ambivalence to awards rather than active rejection, but 3:1 ratio among the
respondents surprised me. I don’t have any figures to back this up, but it
feels like a reflection of the “do we even need awards” sentiments that
circulate in online fora and are debated on YouTube channels around the time of
the CSR nominations announcement each year for the last few, though 25% seems
higher than I would have expected.
I hadn’t thought about this being so prevalent a
view among publishers, otherwise I would have built another question into the
query letter. I suspect that this may be an artifact of the still relatively
small collection sample. It’s a subject that perhaps should be discussed, but while
it warranted noting, it’s not the concern of this study.
Being an independent company, the impact is minimal. Mainstream brands
always attract more attention than we do, regardless of how the products
compare.
- Carlo Amaddeo, WBS Games
I should add that none of the comments were hostile
to the idea of wargame awards or to the CSR or Summit Awards that were singled
out as examples in the questionnaire. Four respondents made further comments questioning
the architecture of the nomination selection system but these comments all came
from publishers who had been favourable of the awards generally.
Nicola Contardi from Europa Simulatzioni questioned
the opacity of the process: “Value proposition would
benefit from an increase of transparency in the overall process of assigning
awards. What are the criteria by which a game is entered a short list? Who are
the persons assigning the awards and why? What is the process granting that the
best title wins the award?” Mr Contradi was ultimately positive regarding the
award process, adding, “While my comments above may seem sort of a negative
criticism, I think that a solid system, with transparent and shared rules to
assign awards to wargames in the international market, would be very important
for our community. I thank you for the work you do, and I encourage all of you
to make it always better.”
The awards do provide a platform for discussion of games which is always good for the industry. While I am biased as a small company, the award winning is for the most part reserved for the large companies though the nominations are appreciated by us smaller guys.
- [Anonymous]
Award
nomination
As previously stated, all the companies I approached
had been nominated at least once in the last five years for a CSR Award, so all
respondents were qualified speak to whether the nomination of an award – in
their cases a fairly well-known and well-covered one – had any influence on
subsequent sales of the nominated title.
In regard to my original question, whether awards nominations translate to (even moderately) increased sales, the overwhelming majority of respondents said it hadn’t. Several said “Nil” or “None” in their comments. This wasn’t an option I’d thought to add in the questionnaire, so I combined the results of respondents had identified a slight (negligible) or noticeable because these seemed to be virtually interchangeable in many of the responses. From parsing these responses, it seems like most publishers have experienced some increase in interest or sales of a nominated product, but that the increase is of an order not much more than what could be a statistical margin of error. No publisher indicated a significant increase in sales.
Which brings us to the fourth category. I dubbed
this “It’s complicated” partly because it amused me to do so, but also because
it succinctly captures a recurring frustration among the respondents. Based on
the evidence reported, the biggest complication preventing a potential bounce
in sales of award nominated games is timing. The pattern of production for the
majority of publishers – those who don’t use print-on-demand partners like Blue
Panther – often produce quite limited numbers of games in a single print-run,
sometimes as few as a few hundred copies. A number of companies have some
manner of pre-order system in place that can inform the scale of production,
but the wargame publishing business is still a business, and nobody wants to be
left with a lot of excess stock. This leads to popular games – the games that
might garner an award nomination – often being sold out by the time the
nominations are announced. In the case of the Charles S. Roberts Awards, where
the awards cover games released in the previous calendar year, but the
selection panel does not meet until three or four months into the following
year, the list of nominated games could appear twelve months or more after the
initial release of a title*, plenty of time for initial stocks to have been depleted.
One respondent, Sergio Schiavi of Dissimula
Edizioni, wrote at length about his experience with awards. I believe it
warrants inclusion here in full:
Well, my only experience is with a
title I published a few years ago, From Salerno to Rome [Dissimula Edizioni, 2020];
it did receive a nomination, but by then I had sold most of the copies I'd
produced in the game's three print runs, about 1,600 copies in total. For
several years, I've been asking people who want to pre-order a game or get
information about a title how they found out about it; most of the responses,
especially on FSTR, are because they saw a video on YouTube or were talking
about it in forums; no one ever mentions the awards to me. But I don't know if
it's a significant sample, there were about 20-25 responses of this type in
four years.
My first title, Radetzky's March [Dissimula Edizioni,
2018], won an award in France for best game of 2019; even then, it had no
impact on sales of the first edition, as all copies sold out well before the
award. I don't know if it had any impact on the second edition [Dissimula Edizioni, 2023], which I produced five
years later.
A leitmotif among respondents from both the positive
and ambivalent camps was the assumption that larger game companies would enjoy
an outsized advantage from award nominations compared to the negligible impact,
if any, for smaller publishers. This was an opinion offered by a full quarter
of the survey respondents, and all of these were self-identified small
publishers. I’m grateful for how many companies did respond to my query
email, but this is where a larger dataset would have been useful.
Award win
The majority of respondents hadn’t won an award, or
at least hadn’t won a Charlie**; using CSR nominees as a research pool had the
unintended effect of focussing the many of the respondents’ comments on those particular
awards rather than wargame awards in general, I don’t have a clear idea of what
other wargame awards are being bestowed these days (if anyone would like to
highlight an regularly offered award from their region or country, please
contact me). Of those publishers whose nominated game had taken a gong, nearly
all said that the award still translated to a negligible lift in sales, roughly
equivalent to if they had only received a nomination. Some publishers were
quick to point out that this was in all likelihood a problem of timing (as discussed
above) rather than lack of influence.
There is also the problem for companies outside the
United States of actually having their product available in what is seen as the
largest market for wargames. Florent Coupeau of NUTS! Publishing wrote:
[The] Charles S. Roberts Awards or
the SDHistCon Summit Award; the votes mainly come from US players. Same for BGG
awards. No problem with that, they were created by US organizations. But that
obviously influences the final results. Now that is democracy, there are more
US wargamers anyway :-)
But even though the board members
include more and more non-US games in the final round, the US players don't get
access to these games as easily as US games. So, they don't know what they are
and cannot evaluate them. Thus they don't vote for them. Fairly understandable.
Only one respondent advised an award win had
translated to an identifiable increase in sales. Walter Vejdovsky, founder of
Fellowship of Simulations, had already established himself as a game designer
with his Eagles of France series, released through Hexasim. Mr Vejdovsky launched his own company,
Fellowship of Simulations with the release of Verdun,1916: Steel Inferno (Fellowship of Simulations, 2020).
The game was nominated for and won three Charles S. Roberts Awards for 2020,
including Wargame of the Year and the James Dunnigan Award for Design
Excellence. Mr Vejdovsky wrote, “CSR awards were really useful when I won
the best wargame of the year awards in 2020 for my first published game. a
great support for recognition of a small publisher.”
Conclusions
I don’t think anybody in or adjacent to the industry
would be surprised by anything brought out in this post. Awards aren’t like
house magazines, they are not in the service of a single publisher. Awards
are given as recognition for some kind of achievement from the vantage of an
audience, usually the customers.
It warrants noting that the European publishers (and
the one Japanese respondent) were generally more willing to speak openly in
their comments than their US counterparts. I don’t want to try to read too much
into this; it may be reflective of different countries’ business cultures
generally.
Looking back on the project, I think the topic would
have been better served if I’d broadened the pool to include publishers outside
of the fairly reductive CSR Awards nominee group. I chose this group so I only
had around sixty companies, instead of the 100 or more it may have potentially
grown to if I’d expanded my starting pool. Reaching out to a broader base of
publishers would have given those who haven’t ever been nominated for an award
or have received a nomination of something other than a CSR, to express their
ideas of that awards represent to publishers. It would be easy enough to direct
those who hadn’t had a nomination to skip to Question X. Also, I think if I
were to do this again, I’d look at using an online survey option like Survey
Monkey, although I’m really not sure if that would translate to a bigger
response pool.
The results are a little skewed in the response;
while I received responses from many more publishers than I dared hope, most of
these were from smaller outfits, many of whom have never won an award in spite
of, in some cases, repeated nominations over several years. This is probably
roughly proportionate to the production landscape in wargaming; for most indy
publishers it’s a cottage industry that many run in their spare time while
working a straight job. I don’t want to speculate on why companies with a
larger industry footprint tended not to reply (except to say that answering dumb
questions from someone with no established credibility in the wargaming landscape
is probably a waste of time), but I suspect two or three responses from the
“other end of the industry may have made for more interesting reading.
Final thoughts
I’m not a publisher, so I can only go by the
information presented here and conversations with people involved in the
industry. That’s why I wanted to run this survey and try to parse some meaning
out of the results. I’m under no illusions that this will be on any use to
anyone, but it may ignite a conversation somewhere.
As a consumer, the value for me in awards like the
Charlies is awareness-raising. Each year there is at least one game that I hadn’t
heard of, often released by a publisher of which I wasn’t aware. And in nearly
every case, that game is a nominee, not a winner.
To be fair, I’m not contributing to any bump in sales,
negligible or otherwise; usually I won’t get to purchasing a game I hear about
through an awards nomination because I operate on a fairly tight budget,
usually allocated four or five months in advance, and I put the same amount of
research into an award-nominated game as I would any other, but I think I’ve eventually
bought at least one game – often two – from those nominated in the CSRs from
the last five or six years that otherwise may have passed me by.
I’ve written in the past about how important I
believe it is to recognise the creators of the games we enjoy and how grateful
I am for the continued existence the CSR Awards in particular. I feel like I
need to reiterate that this exercise was born out of personal curiosity regarding
the impact these awards have – or don’t have – on the business-end of the
hobby. I’d gone into this project (nearly three months in the making – I started
mailing out questionnaires in early July) hoped to get better news about the effect
a game’s sales prospects of a nomination for a high-profile award, but the
evidence says otherwise.
In spite of this, the majority of publishers still think
positively of wargame awards and support their continuance. They’re not uncritical
of the way the sausages are made, but they still enjoy the grill.
The final word should go to Kevin Zucker of Operational
Studies Group. Mr Zucker – like several others – was generous with his time and
comments, indulging all of my follow-up questions. He closed this: “There is no
direct correlation, although it is never a bad thing to win an award. The real
award is people playing and having fun.”
As I added these comments to my spreadsheet, a follow up message arrived:
“The ultimate award is a game map in tatters and the ink worn off the Napoleon counter...”
* This isn’t meant as a criticism of the structure or
management of the CSRs. It’s simply recognising
that no matter how a sample period is structured, it is probably going to
affect somebody adversely. This is unfortunate, but there is simply no way to
create a perfectly fair system. To paraphrase Churchill, it’s the worst system,
barring all the other ones.
** Of the twenty-four respondents, eight publishers had won
at least one CSR Award in the last five years.
Appendix 1: Charles S. Roberts Award nomination and award distribution over the last five years (2020-2024).
No comments:
Post a Comment