Author: johnob21

Evaluating artist-led projects is persistently uncomfortable work – and I’ve done a lot of it in recent years: artists in residence, artists in health settings, artists and climate, artists embedded in community systems, in bureaucracies, etc. All this work that artists do sits slightly sideways to the frameworks we are expected to use to account for public value. Once the word evaluation enters the brief evaluators reach instinctively for outcomes, indicators, deliverables.

The problem is that Artists reach for something else entirely.

This mismatch is not a technical problem by the way. It’s a conceptual one.

So, bear with me. The problem is that most evaluation frameworks assume a linear logic: inputs produce activities, activities produce outputs, outputs produce outcomes. (Who doesn’t love a good logic model). The assumption then is that the artist enters this logic as a producer of things – artworks, workshops, events, moments of participation – which can then be measured for impact. But this assumption, this logic misses what artists are actually doing in these contexts. It misreads the nature of their labour, and therefore misunderstands their value.

Artists in these roles are not primarily producers. They are researchers. Arguably art is the original form of all research.

Obviously not researchers in lab coats or with spreadsheets (though sometimes those appear), but researchers in the original sense of the word: people who re-search a situation. They look again. They look differently. They attend to what is usually overlooked.

Like all good researchers, artists begin with relationships. They embed. They listen. They notice patterns, tensions, contradictions. They collect fragments: stories, gestures, materials, overheard comments, institutional habits, silences. They connect these fragments across boundaries that usually remain separate – between departments, disciplines, people, timescales, ways of knowing.

Amanda Palmer once described her work as “collecting, connecting and sharing”. That phrase captures something essential. It is not extraction. It is not production. It is sense-making.

This is why artists are particularly adept at working in complex systems. Health systems. Local authorities. Climate responses. Communities shaped by layered histories and power relations. These are not environments where change moves in straight lines or where causality can be neatly traced. They are living systems, shaped by feedback loops, informal practices, emotional undercurrents and inherited assumptions.

Artists sense complexity. It’s part of the toolkit. Artists work with ambiguity rather than trying to eliminate it. Artists notice where language fails, where processes contradict stated values, where people behave differently than policies suggest they should. They feel the system as much as they analyse it.

From an evaluation perspective, this creates a problem only if we insist on the wrong question. If we ask, “What outcomes did the artist produce?” we will always be dissatisfied. The answer will feel partial, anecdotal, resistant to quantification.

If instead we ask, “What did the artist surface?” the work becomes visible.

What relationships shifted?

What conversations became possible?

What assumptions were disturbed?

What new ways of seeing entered the system?

What world was reflected back at us?

What did we learn about ourselves?

These are not soft or secondary effects. In complex environments, they are the conditions that make any durable change possible.

The mistake we repeatedly make is to treat the artwork as evidence of impact rather than as a mode of evaluation in itself. We try to translate the work of artists into reports, logic models or metrics that sit outside the practice itself, as if the art were merely illustrative.

But the art is the evaluation.

The artwork – whether it takes the form of an installation, a performance, a score, a temporary intervention, a set of instructions, a story, or a social process – is how the research is shared back into the system. It is a proposition. A reflection. A reframing. It carries data that cannot be reduced without being destroyed.

This does not mean abandoning our critical rigour. On the contrary, all art work is an intense methodological discipline: careful observation, ethical attention to participants, iterative testing of ideas, responsiveness to context. What art and artists resist is not rigour, but reduction.

The push by funders and policy makers to frame art in terms of outcomes reflects a deep institutional anxiety rather than a concern for genuine accountability. Outcomes promise control. They suggest that if we design the right intervention, we can predict and manage change. Artists, by contrast, reveal how little control any of us actually have – and how much depends on relationships, trust, timing and openness.

This is uncomfortable knowledge, particularly for organisations structured around a faith in certainty and compliance. But it is precisely why it’s important to understand that art is research and that is why it is so valuable.

So what have I learned from recent evaluations of artist led projects. Evaluating artist projects requires a shift in stance. Less auditing, more interpretation. Less counting, more attending. It requires evaluators to read artworks not as decorative extras, but as analytical texts – dense, layered, situated.

It also requires commissioners and funders to accept that not all value arrives on schedule, or in forms that can be anticipated in advance. Some of the most significant effects of artist-led work are delayed, diffuse or indirect. They show up later, in changed practices, altered language, new alliances.

If we are serious about learning from these projects – not just justifying them – we need evaluation approaches that mirror the artists’ own intelligence: relational, systemic, reflective.

Artists are not delivering outcomes to a system. They are helping the system understand itself.

And in an era defined by complexity, that’s probably the most necessary research of all.

At some point every arts/cultural organisation and probably every artist is asked to produce a “plan” Depending on who’s doing the asking you might be asked for a Policy, or a Strategy, or a Business Plan, or a Business Model, or a Funding Model, or a Revenue Model, or a Resource Plan or – my personal favourite – an Implementation Plan. Sometimes you’ll be asked for all or most of them at the same time, and other times you’ll be asked for one thing but the person doing the asking actually wants something else (this is way more common than it should be).

So, in the interests of sanity I offer this simple taxonomy to ease the pain of dealing with the deep anxiety that planning tries to mask.

So, let’s start with Policy (which is a kind of plan). A policy must have a clearly defined “object”; it must have a precise purpose – something it wants to do to or with the “object”; and it must have a tool, or tools, with which to achieve the purpose. So, for example, “affordable housing” is kind of a clearly defined object; making it widely available is a fairly precise purpose; and tools could include tax incentives, funding, legislation, government loans, public private partnership, shifting the county onto a Welfare State model etc. Culture, on the other hand, is not a great policy object because it’s really hard to define and is not really a thing in and off itself but the result of lots of other things. Making policy for results or outcomes (as opposed to the things that produce or contribute to the results or outcomes is considered in the policy world to be a really bad idea).

Strategy. This frequently gets confused with implementation (and Policy, and Business Planning, and nearly everything else) and it is absolutely not about implementation. If somebody asks for your implementation in a strategy conversation try and distract them with something shiny. Strategy is the answer to the questions “Why”, “What” and “How”. It’s really that simple. Why? because there’s a need or a gap or an opportunity or a belief. How? In collaboration, or in partnership, or sustainably, or slowly, or inclusively, or ruthlessly or profitably. One of the principal reasons for making strategy is that we can answer the questions “why do you do what you do?” (or why your organisation does what it does?). The answer to the Why is the vision part of the strategy (and a little bit of the mission). And when you give the answer to the question why, people ask you “how will you do that?” and you’ll list off the values (and a little bit of the mission) because values are how you will behave as you do what you do to achieve what you want.  Strategy also answers the What question (the mission bit of strategy), so for example you can say that you want to make sure that everybody has a place to think and talk about their lives so they can imagine a better society together (The vision bit that answers the why question), and you will do this by producing community plays throughout the country that provoke and encourage debate (the mission bit of the strategy that answers the what question) and you will do this in “collaboration and partnership” with communities, local governments, national agencies,  in a “sustainable” and “inclusive” way (and that’s the  values bit of strategy, that answers the “how” question). Essentially, that’s it. Draw a line under it and walk away

Business Plan. The demand for these increased over the course of my career, primarily due to the Rise of New Public Management which was really just the triumph of market capitalism over the ideas of citizenship and public service. A business plan needs to have some solid research to be credible and it needs to express the strategy in numbers. At the most basic level The Business Plan needs to demonstrate that there are enough people in a very specific place or places, willing and able to give you (as opposed to somebody else) enough money on a regular basis so you can carry on doing what you’re doing. The most common error in Business Plans is confusing your product or service with your Business: the latter is how you will make money from the former so your business plan needs to describe what your product/service is , and it needs to do this very clearly and very concisely. Then you need to use the better part of your plan explaining:

• why people in your potential market area might like it/need it/want it,
• how many people in your potential market area might like it/need it/ want it,
• how you will get their attention and custom,
• how much they are prepared to pay for it on a regular basis,
• and what it will cost to produce and deliver the product or service.

The numbers need to be real and verifiable, and the identified need must be supported by strong evidence. The Business plan also needs to say who’s on the team, why they’re on the team, and what value they bring to the Business.

So, a business plan for an arts organisation, for example, needs to describe what the product or service is (e.g. A Local Festival); why it’s needed in this place (e.g. there are no festivals in the local area, there is a local appetite for a community event, and the demographic evidence suggests an arts festival would be supported), there’s a population within a 30minute drive of 100,000 so in all probability a potential market of 20,000, the demographic evidence suggests they are able/prepared to pay €100 for a festival event once a year so potential market value is about €2 million. Now you won’t get all of that ( because of competition and capacity) and you certainly won’t get more than that. You will need to outline a realistic marketing plan (audience segmentation, user personas, inbound and outbound marketing tactics etc.). Then you need to set out your operational costs, your programme costs, and your marketing costs, your risk management (and a summary of the strategy), and then you need to include the team and the organisation structure (who’s responsible for what) and Bingo! Business Plan done. At no point in the business plan do you need to set out the specific programme details or festival schedule. Because you don’t know the details at this time and the reality is that the details of delivery (implementation) change when you start to deliver – the unknown unknowns start to kick in.

There’s lots of popular tools you can use (PESTLE, SWOT, SOAR, 5 Forces, Ansoff’s Matrix, Logic Model, Forecasting, Blue Ocean, 5Ps etc. etc.). The trick is to choose the appropriate tool and that tends to be the one you understand. Remember that most of these basic business tools are mostly just ways to categorise and present information, they don’t do anything by themselves, and they are frequently misused and misunderstood, even by those who ask you to use them.

The business plan is a way of checking feasibility for yourself, and is designed to be read by people who can support the venture – funders, investors, other stakeholders etc. – to convince them that this is a good idea.

The Business Case has a lot of similarities with the Business Plan, except that it’s usually an internal document concerned with a single project in the business and not with the overall business. For example, if the organisation wants to invest some of its resources  (time, money, talent) in a specific project or venture (buying a dedicated property, investing in audio-visual equipment for digital broadcast, allocating a greater percentage of its overall budget to marketing, spending resources on a Van, etc) then it can be worth preparing a business case that says why this allocation of resources is a good idea, how it contributes to the business, how it aligns with strategy, what it will deliver for the organisation, and what kind of return on investment is expected. So for example, buying a van will increase the reach of the work and therefore increase the potential market size, its cheaper than hiring a van over the medium term, it can function as a mobile branding and marketing tool, and we will save money on transport hire over the medium term and we can rent it out to other local organisations so it can pay for itself and contribute to the revenue model.

The Business Case is usually an internal document focused on a specific element of the overall business and allows you to clarify your thinking and demonstrate to relevant people (mangers, CEO, Board of Directors, key funders etc) that the idea is a good one. Sometimes, when making a specific application for for funding (e.g. arts council or local authority) for a very specific project or capital support you are effectively preparing a business case, or a national agency looking for funding for a new IT system or new staff will prepare a Business Case for its relevant Department.

The Business Model. This one can lead to a lot of confusion but essentially, it’s a way of simplifying – usually in a visual format – what your business is (think of it like a full body X-ray). Its useful to have one on the wall of your office, and it should be built on a white board, or with post its, or with bits of paper sellotaped to the wall.  It summarises the key business offering (that tends to be the Mission and Vision of the strategy), the resources you need, the activities you need to engage in, the partners you need to work with (suppliers, funders etc); it also summarises the customer segments, the kind of relationship you have with them, and how you communicate and deliver to them. It summarises the costs and the income.  It’s a really useful way of explaining to yourself and the team what business you are actually in and what that business actually does, and it’s a really useful way of spotting opportunities and challenges on an ongoing basis – once you keep it updated!

Funding Model and Revenue Model. Again – people get these confused (including people asking you for them, which doesn’t help). So here we go:

The Funding Model: this essentially lists the sources and amounts of funding necessary for the operation of your business.  In an arts context we can include all the sources of funding (money that funds what we do), so we’ll probably include Arts Council, Local Authority, Debt (bank loan or overdraft), Total Sales ( the sum of tickets, bar and cafe, art sales, rental etc.), and other sources including ETB, Pobal, CE schemes, and sponsorship or donations.  It’s a really good idea to express these as both cash amounts and percentages so you can see the relative weight of each funding source. Building a Funding Model for yourself allows you to have a conversation around the funding risks (which is really the main point of it), and on overdependence on specific sources, on identifying additional sources, on sources that can deliver more etc.

The Revenue Model: This is a deep dive into your Total Sales income: it’s about  understanding how you “earn” money. It looks at the ticket prices to understand which price delivers more income, and which customers deliver more income. It’s about understanding that if 20% of your customers deliver 80% of your earned income you need to ask some questions about the other 80% of customers. It’s about listing all the sources of earned income (ticket sales, commission on art sales, bar sales, merchandising, copyright, guarantees, booking fees, rental etc.) to identify weaknesses, risks and opportunities, dependencies etc. I’m always surprised that organisations that provide artist support and development for example don’t secure a percentage of ongoing rights in the work. The income will be – in most cases – small or negligible but in some cases can be significant over time.

Implementation Plan. I don’t know why but bureaucrats love the implementation plan. Its possibly because so many of them have come from engineering, business, and project management backgrounds. Implementation plans are funding dependent. You can’t create an implementation plan if you don’t know what the funding will be (actually you can, but it’s kinda pointless). The business plan, or business case, or business model will give you some ballpark numbers around the funding need. These numbers are the basis of the ask. When you have an actual number in response then you can start an implementation plan.

Implementation plans work best if they are project specific because they answer the question “how are we going to deliver this?” Implementation plans are project management tools so they need to contain work break-down structures, timelines, dependencies, (these things can happen at the same time, but this other things have to happen sequentially), milestones, phases, resource requirements and allocations, RACI charts (who is Responsible, who is Accountable, who needs to be Consulted and who needs to be Informed), detailed costings and final overall budgets. They need to have a live risk management, monitoring and control process, and they need to have an agreed change process (what happens when the world doesn’t behave according to our plan). Implementation is not the starting point. If there was a planning sequence then Implemention comes at the end.

Planning as Metaphor. Planning is all about reducing anxiety. The different plans address different anxieties and they are designed to be read by different groups of people. In this sense plans are performative, like magic powers in a Live Action Role Play.

Plans (as opposed to planning) are also about power, about the dominance of the “rational” over the “emotional” or the “creative”.  When people ask you for a particular kind of plan, ask them what they want to see in that plan, because there’s a 50/50 chance they’re asking you for the wrong thing, and what they want to see will tell you what their anxiety looks and feels like.

Having said all that, planning is useful and planning is necessary, however there is no right way of doing it, and all plans inevitably change the moment we try to realise them, the moment they come into contact with the messy complexity of the world.  

Let’s finish with a quote from Ritter’s “Dilemma’s in General Theory of Planning” – one of the great meditations on planning.

“In turn, and equally intractable, is the problem of identifying the actions that might effectively narrow the gap between what-is and what-ought-to-be. As we seek to improve the effectiveness of actions in pursuit of valued outcomes, as system boundaries get stretched, and as we become more sophisticated about the complex workings of open societal systems, it becomes ever more difficult to make the planning idea operational… The information needed to understand the problem depends upon one’s idea for solving it. That is to say: in order to describe a wicked-problem in sufficient detail, one has to develop an exhaustive inventory of all conceivable solutions ahead of time. The reason is that every question asking for additional information depends upon the understanding of the problem – and its resolution – at that time. To find the problem is thus the same thing as finding the solution; the problem can’t be defined until the solution has been found”. (Dilemmas in a General Theory of Planning, Ritter & Weber, 1973)

So there I was, late one evening, a little bit bored and checking through email. One email told me that I should check out Google’s NotebookLM. Apparently it could turn some rough notes – or even a full essay – into an actual podcast conversation. Yeah. Sure. So I uploaded some notes I’d been working on, some rough ideas, from a previous lecture and it produced this. I give you AI working off my notes and explaining me to myself

Now, it didn’t interpret all my notes accurately, and it certainly misses a lot of nuance, but hey – what a conversation. Enjoy!

And by the way, the image on this page was created by wordpress’ AI. Mad!

There’s an increasing number of courses at third level in Ireland that have the words “Design Thinking” in their title. I should know – I’m currently on one of them. Its become quite the hot topic across business and government recently. The Department of Public Expenditure, NDP Delivery and Reform just published its Action Plan for Designing Better Public Services – A Roadmap for Embedding Design in the Public Service 2024 – 2025.

Minister Pascal Donoghue, states in the introduction “Through our collective efforts, we are charting a course towards a more dynamic, inherently responsive, people-centric future for Ireland. I want to strongly encourage public servants at all levels to adopt design approaches in how they work, and I look forward to seeing the outcomes and experiencing the impact”.

He also states that “The design tools, processes, principles and mindset outlined in this document can help us work in new ways, ask different questions and take on our challenges with empathy, insight and imagination”.

Its been something of a shock to discover that the Design Thinking mindset and so very many of its tools and techniques are essentially how artists think and what they do. Sure, there are some tweaks in what things are called and some formal diagrams and tempates but fundamentally Design Thinking is about returning artists and artistry to the heart of planing, development, decision making, strategy, policy etc.

Lets just take a look at some of the key phrases in design thinking and see how they map onto the work of artists:

1. Innovative Problem Solving: Artists see problems from multiple perspectives almost as a matter of course. Its hard to imagine not seeing the world this way. Its what allows artists generate innovative solutions/responses/ideas that would not be immediately apparent through rational thought processes.
2. Visual Thinking: Artists are adept at communicating complex ideas visually (and aurally!). That means they condense complex concepts into compelling images, metaphors, scenarios, making them more accessible and engaging, providing focus but allowing for interpretation.
3. Empathy and Understanding: This is the bread and butter of the artists existence. Sometimes its the curse. They are (mostly) inherently empathetic because they draw inspiration from human emotions and experiences. This model of empathy is central to design thinking! Design Thinking asks policy makers and strategists and planners to prioritize understanding the needs and desires of people (end users, audiences, communities). Design Thinking claims that by putting ourselves in their shoes, we can create solutions that truly resonate with their needs. Artist, anybody?
4. Iterative Prototyping: The idea of rough prototyping and many iterations of the same idea is another key principle of Design Thinking and the daily work of the artist. Test and test again refining our strategies, our sentences, our colours, until they meet the desired outcomes.
5. Attention to Detail: Hello! Artists thrive on detail!

Before I became a consultant I was a theatre director and acting coach, so lets look at Design Thinking from the perspective of that particular art form.

1. Empathy Building: Check. Actors need to understand and embody different characters’ perspectives and emotions, and they need to be sensitive to the actor they’re working with. It can be an overload from time to time but building characters (Design Thinking calls them “personas”), and understanding the arc of a character (Design Thinking calls it “journey mapping”) is literally what we pay actors to do.
2. Role-Playing and Scenario Building: Role-playing and scenario building. Check. Actors improvise situations all the time to understand context, structure and motivation. If you want someone to accurately and creatively simulate user experiences and anticipate their reactions to various solutions call an actor. By extension most organisations can be understood as great big dramas with multiple characters and motivations, with subtext and subterfuge. Actors can step into the the shoes of different stakeholders, and tells us how that person actually sees and experiences the world!
3. Storytelling: Check! Acting is fundamentally about storytelling, and it conveys complex ideas and emotions in a compelling and memorable way. Actors and theatre folk can use storytelling techniques in design thinking and create narratives and immersive experiences that resonate with stakeholders.
4. Creative Collaboration: Check! A rehearsal room , a film set, a live performance is built on effective collaboration and trust among actors, directors, designers, and crew members. These people understand the value of of listening, improvising, and building on each other’s contributions to achieve a shared goal. All of this is Design Thinking in action.
5. Embracing Uncertainty and Iteration: Check. A good production will spend 80% of its time asking questions, researching, experimenting, refineing, and adaptating based on constant feedback and about 20% of its time pulling a finished product together. Solutions emerge in this work and frequently they are not solutions but resolutions. The die hard Design Thinkers will get what I mean by that. The point here is that this iterative process mirrors the iterative nature of design thinking, where solutions evolve through continuous prototyping and testing. Theatre artists – and all artists – spend a lifetime embracing uncertainty and view failure as an opportunity for growth – because there’s not really another choice.

So what’s my point? It seems like all those odd behaviours and ways of thinking that led family and friends to worry about you and ask questions like “when will you get a real job? ” or “when are you thinking of joining us in the real world?” are finally being recognised for what they are: necessary, complex, valuable and, above all else, quintessentially human.

Don’t get me wrong, I’m not suggesting that Apple or Google or the World Bank are going to employ you to paint a picture or write a novel! At least not anytime soon. But the landscape is changing and if you chose to bring your artist “mindset” to market then there are enormous opportunities in front of us now. Yes, you will need to pick up some new tools and some new terminology but the talent, the “unique perspective“ and the “mindset” are all in place.

LIkewise if you’re reading this and you want to embed Design Thinking in your organisation then remember that artists have a head start on this, a deep understanding of its musculature and nervous system. Maybe it is time to test Richard Boyd Barret’s plan for every Government Department to employ artists!

I’m going to leave you with a quote from one of the Godfathers of Design Thinking, Bruce Mau, who writes in his last book The Nexus “In a time of increasing complexity and uncertainty, we need a new way of thinking to innovate and lead into the future. Blurring the boundaries between the largest domains of human creativity — art, technology, and science — leads to a dramatic augmentation of thinking spaces and a wealth of possible ideas. Myths and stereotypes in each field, however, have stood in the way of innovation …The key to succeed in reaching THE NEXUS, the place where these modes of thinking converge and synergize, is to be open to new ideas and to seek inspiration from domains outside our own.”