Discover Intelligence & Interview and Subscribe to the Newsletter!
After so many topics, it was time to face one of the structured analytic techniques, also known by the acronym “SATs” (where the “s” is the plural). Actually, when I try to explain to my mother (ah, the mothers!) what intelligence analysis is about, I use SATs. Well, not for analyzing her, but for giving her a concrete example of what intelligence analysts do. All we need is SATs, according to many. But the research in the intelligence studies shows that SATs are not so widespread, their benefits are not so measurable, and ultimately (you will discover in this interview) they are not even so widespread. All the leading intelligence scholars from different corners of the world tackled the issue and, still, there is no universal agreement. Whatever their pros and cons, whatever they are, this is a crucial topic and, I believe, we all must know what they are (if we deal with intelligence). Exactly, for this reason, I thought it appropriate to let Alexei Kuvshinnikov speak about them. Indeed, Alexei is a passionate and professional SATs user, a member of the International Association for Intelligence Education (IAFIE), and, as you will discover, a promoter of SATs use. As a professional expert active in criminal investigation and narcotics for international institutions, and a teacher, he argues for the need for SATs for limiting biases and cognitive pitfalls. Considering his long experience in the field and his knowledge of intelligence methods, Alexei was an ideal referent for talking about this interesting topic. I don’t want to spoiler more, but if you are interested in intelligence analysis, this is something for you. It is then with my distinct pleasure to publish the interview on Scuola Filosofica – for those who don’t know it yet; it is one of the leading cultural blogs in Italy. In the name of Scuola Filosofica Team, our readers, and myself, Giangiuseppe Pili, Alexei: thank you!
1# Professor Alexei Kuvshinnikov, let’s start with the basics. How would you like to present yourself to the national and international readers and Philosophical School (Scuola Filosofica)?
Dear Giangiuseppe, thank you very much for the compliment, but I have to decline it. Being just a titleless lecturer with no academic qualifications beyond a Master´s degree, I have no pretence of belonging to the academia. Getting a taste of reality leads to getting a taste for reality, and graduate students can only benefit from it, that´s my firm belief. Accordingly, I teach SATs not as a science but rather as a tradecraft. You see, from an academic perspective, there is no difference between the academia and the real world. From the perspective of real-world operators, there often is.
2# How everything started? How did you get interested in intelligence and what is your main expertise in it?
Well, one day in the early 1990s, I saw the light… Back then, intelligence analysis was a closet occupation. Nobody took it seriously and it was an uphill struggle to convince law enforcement agencies to recognize its utility. That was the general feeling and feedback I got from the ranks of IALEIA, the International Association of Law Enforcement Intelligence Analysts, which I joined about that time. Everything changed with 9/11. The penny dropped. Disasters and catastrophes do tend to stimulate paradigm changes in human attitudes. It was kind of a seaquake that produced a huge tsunami wave with intelligence analysts riding on its crest. Overnight, their market value skyrocketed and analytic skills became a most prized expertise.
At that time, I was working in the UN Office for Drugs and Crime (UNODC) and beating my head against the wall trying to convince my colleagues and bosses of the merits involved in the use of operational criminal intelligence analysis to support counter-narcotics investigations. I knew what I was doing, after all I had trained with some of the best police departments in UK and the USA, but nobody paid me any attention. And then, suddenly, like at the end of “Who is afraid of Roger Rabbit?” the wall collapsed in a heap of rubble and I was in the Promised Looney-Land.
During the better part of the following decade, I was involved in the building up of criminal intelligence capacities on the Balkans and later on in Central Asia helping police departments there to join the analytical mainstream. That is important, you know, because organized crime operates across borders and investigations must also involve police services from multiple jurisdictions. The more departments acquired the capability to participate in the collection, collation, evaluation and analysis of operational information, and the more they shared their analytical products, the more effective counter-narcotics investigations gradually became.
About ten years back I then became drawn to strategic analysis and discovered SATs and the whole analytical universe behind them. That was still pretty near to the Big Bang and the universe was still fairly compact. But it was expanding exponentially and it took me several years of study before I could gradually build a university course that I now have been teaching since some five years.
It has a focus on the use of SATs for strategic threat analysis and forecasting and I have consciously configured it to fit into international affairs rather than intelligence degree programs. Since I know frow experience that the only way to learn SATs is by doing, by practicing, group home projects form the backbone of my teaching method. I´ve also developed a growing number of case studies discussing real-world issues in foreign politics. By the way, one of these involving the application of High-Impact Low-Probability analysis to an invasion of Lampedusa by migrants is a favourite with Italian students.
Well, to be fair, besides a good number of Italians, my students also come from France, Germany, Austria, Slovakia, Switzerland, UK, Russia, USA, South Korea, China, Brazil, Mexico and Libya…
3# You told me that you worked as a senior expert on drugs/OC under projects funded by the European Commission. This is extremely fascinating. What can you tell us about it? And, being an Italian, I must ask about the involvement of the Italian mafia-like criminal organizations role!
In essence, it is a desk job but with plenty of opportunities for visiting remote and inhospitable locations, sometimes under armed escort, to talk to people who are not necessarily on your side as well as all kinds of really shady types so yes, it fits with my idea of good fun.
I had memorable run-ins with some other-worldly characters that evolved into a lasting and trusting relationships. That´s how I came to understand and appreciate the role of “ponchos rojos” (red ponchos) in Aymara communities on the Andean altiplano. They are often portrayed as fearsome and broadly feared Aymara militia, brutal enforcers of tribal discipline, who keep their own counsel. But you know, once they agreed to talk to me, grudgingly first, they also listened to what I had to say. I´ve spent enough time with them to understand their true role and true people´s attitudes towards them, which helped me to better align a set of policy options I was working on with the reality.
Good strategic options are not a product of a high-browed intellectual discharge. They are not invented but rather distilled from grassroots opinions.
Since some ten years my focus is on coca bush cultivation and cocaine production in South America. I served as a long-term resident advisor to the Bolivian Interior Ministry and did short jobs in Peru and Colombia as well. Without going into too much detail, I can say that my responsibilities included strategic threat analysis and futures work. In plain words, trying to figure out, which one of the infinite number of possible futures is most likely to unfold. An objective is to inform government departments in their formulation of proactive strategies for drug demand and supply reduction. To that purpose I used different versions of the Long Matrix for ranking threat actors, econometric modelling based on the New Lanchester Strategy, Multiple-Perspective Utility Analysis for seeking consensus policy options and a bit of game theory for bargaining and search for players´ dominant strategies.
My assessments didn´t always please my local counterparts. You can well lead a horse to water but you can´t make it drink. It´s rather often that not only politicians but just about everybody develop selective hearing. On one hand, it is sufficient just to whisper into the wind what they are eagerly expecting to hear. Humans prefer to believe what they prefer to be true. Judgements that go against their expectations and beliefs they prefer to ignore. You can yell at them through a bullhorn, but they just pretend they don´t hear you. It can be tempting to yield to reality and tweak your own views but succumbing to the satisficing bias is a cardinal sin for an analyst. Most recently, I had this experience in Bolivia in July 2019. In my warning analysis I alerted my Bolivian colleagues about an unfolding future that didn´t bode particularly well for them. Since we were good mates and didn´t talk bullshit to each other, they fondly but firmly told me to mind my own business. Well, my assessment proved to be right – and they became history.
Now, regarding your question about the mafia… To be honest, you being an Italian will probably know more about Italian organised crime than I do. In any case, I am not an expert on this subject. I sure have very fond memories of rubbing elbows – but that was quite some time ago – with DCSA, Direzione Centrale Servizi Antidroga, which is the multi-service counter-narcotics department, and anti-mafia prosecutors. These were amazing people for whom I have deepest respect.
I remember an occasion when I was sitting with one of them in a run-down roadside café in the North of Albania, munching on a Tavë kosi, a local lamb stew, and drinking raki while waiting for the local connection to arrive. I asked what he did during the three years that he had spent under guard in his tiny apartment in the sole company of his wife, leaving it only to go to the court. He thought about it carefully, sipping raki with an impassive face and finally shrugged his shoulders and said: “What did I do? I did children…What else was there to do?”
But I´ve digressed…
Quite recently, though, I´ve stumbled upon the “Mafia connection” through my teaching. On a recent course, two Italian students, Pierpaolo Proni and Luca Urciuolo teamed up to do a project that involved using SATs to rank threat actors engaged in migrant smuggling to Italy. And they discovered in open sources some interesting evidence about cooperation between “traditional” Italian organised crime and foreign criminal groupings that was new to me.
So, I did a bit of my own research. It showed that both in Sicily and on the Italian mainland “traditional” organised crime – Cosa Nostra, Stidda, ’Ndrangheta, Santa Corona Unita and Camorra – were collaborating and forging alliances with “foreigners”.
In Sicily, Cosa Nostra profits from state subsidies by using fronts to obtain contracts for managing the accommodation of migrants and refugees. Besides that, local cooperatives that operate many of migrant reception centres throughout the island subcontract supplies to companies controlled by the mafia. It also preys on migrants and refugees by paying meagre rates for their labour or coercing them to do smuggling, drug-running, fighting the turf wars or prostitution.
But that´s hardly anything new, right? What was new to me was that Cosa Nostra did not seem to be very possessive about its home turf. Apparently, it allows foreign criminal operators to “wet the beak” in the Syracuse province. And in the Sicilian heartland, Enna, Palermo and Trapani, it coexists with the Nigerian Vikings gang and the remnants of Black Axe organisation. On the mainland, Camorra and Nigerian criminal clans cooperate in the Caserta province of Campania.
The whole business of sending migrants and refugees across the Mediterranean to Italy is, of course, just that – a flourishing and highly lucrative criminal business controlled by organized crime groups and criminal organisations.
The control of migration by organised crime can produce sinister consequences. Consider the COVID-19 outbreak in Europe. It began in Italy. To be more precise, in Lombardy. Some of the hardest hit provinces were north of Milano, Como and Varese together with Mantua a bit to the south. That´s where hundreds of sewing factories are located. A fair share of their 100,000-strong labour force is made up of Chinese migrants. Wuhan is a Chinese equivalent of Lombardy with abundance of skilled seamstresses and tailors that fuels the ethnic Chinese community in Lombardy. While many of these factories are perfectly legal, there is also a fair number that are run by organised crime – ’Ndrangheta in particular – underground and employ labour force that includes illegal migrants. Chinese returnees brought the COVID-19 virus from Spring Festival celebrations in Wuhan and since many lived and worked below the radar, the spreading of infections remained undetected for quite some time. When the authorities realised that something strange was going on, it was too late.
But I have again digressed from the focus of our conversation, sorry about that.
4# Let’s come to the core of the interview. How would you define Structured Analytic Techniques? Why are they important?
To answer your last question in one sentence – SATs is the only known remedy against cognitive biases in analytic work. That´s worth its weight in gold, biases are responsible for most analytic failures.
The human mind cannot cope directly with the complexity of the world. Rather, we construct a simplified mental model of reality and then work with this model. Such – inescapable – mental models are called mind-sets. A mind-set is akin to a lens through which one perceives the world – and this view is inexorably slanted by biases.
That´s the key thing. Mind-sets are unavoidable and they are inevitably distorted by biases. Yours, mine, everyone´s. It is a characteristic of our species. An absolute majority of people have no clue that their vision of the surrounding world is a crooked one. And the minority who are aware of it can´t do anything much about it except, well, being aware. Cognitive biases are built into human thinking process and are highly resistant to conscious efforts to overcome them or compensate for them. They will persist despite an analyst´s best efforts to the contrary. Biases are at work mostly at the subconscious level and we cannot control them nor even observe them at work.
You can train yourself to think about how you are thinking. It´s a bit like having a split mind that exists simultaneously at two levels. You kind of follow your own reasoning from a distance. It may sound a bit, well, crazy, but it´s just a reasoning skill and with a bit of practice and experience you can train yourself in it.
The point is, while mastering that skill could somewhat help you to identify your biases, even that won´t be much of a help in controlling them. Nothing will. Except SATs. A person´s reasoning is a black box. SATs, at the very least, will open it to external scrutiny and critique. That´s one big step forward towards identifying judgements that have been soiled by biased reasoning. And if SATs are applied working in a team, which is their ultimate purpose, individual biases will kind of reciprocally throttle each other. Regardless, one can´t expect that a team´s judgement will ever be completely free of biases. But the distortion will be greatly reduced as biases get weakened by exposure to each other.
SATs represent a set of simple yet powerful reasoning-enhancing tools. They were developed by the intelligence community and over the past decade or so released into the civilian analytic domain. So, SATs – by definition – are simple and repeatable techniques that aim to produce repeatable and comparable results. According to Heuer, they can be grouped into eight subdomains – Idea generation; Scenarios and indicators; Hypothesis generation and testing; Assessment of cause and effect; Challenge analysis; Analytical conflict management and Decision support.
Led by the pioneering work of Richards Heuer Jr., Katherine Hibbs Pherson and Randolph Pherson, many brilliant minds have contributed to SATs rising in prominence. I stand on the shoulders of giants!
The current SAT surge comes at just about the right time. Look at today´s strategic threats. They reflect an unprecedented degree of complexity and fluidity. They increasingly involve non-state hostile actors and rogue groupings that have displayed “transformer” capability adapting to international efforts aimed at denying them operational freedom. Plus, the unfolding of globalization has led to a dramatic increase in the diversity of outcomes creating a plethora of possible futures.
5# There is an endless debate around Structured Analytic Techniques. First, are they widely used? Is it possible doing intelligence analysis without using some Structured Analytic Techniques?
It´s hard to say how widely are SATs used. It´s not necessarily advertised. They seem to have been as good as institutionalized in the US intelligence community. They may be used to some extent in other “five-eyes” intelligence agencies. Some of them have been adopted by Europol and hence applied by police services of EU Member States. Beyond that, it´s hard to say.
In Europe, I guess it´s basically up to individuals to carry the flame. When doing jobs for EU Delegations in South America, I used SATs not because they are adopted by EEAS – they are not – but because they produced better results. None of my desk officers in the Delegations was knowledgeable about SATs, but to their credit, up to the last one of them they listened to my briefings with interest and attention and were easily satisfied with my analytical products.
It´s for sure possible to conduct intelligence analysis without applying SATs. After all, that´s the way it´s still mostly done. This is a road that is strewn with carcasses of analytical failures but these, more often than not, are just shrugged off. Products of strategic intelligence analysis often serve as basis for taking decisions that can impact the life of thousands of people – but rarely affect in the same degree analysts and decision-takers themselves. Using Taleb´s expression, they don´t have “skin in the game”. If you are a surgeon and amputate a wrong leg, your future will look rather bleak indeed, wont it? But if you are an analyst and produce an erroneous assessment that justifies the politicians´ intention to, say, unleash a war, so what? A bad call, but us humans are prone to making errors, OK?
No, it shouldn’t be OK, I believe analytic failures by commission or omission shouldn´t be accepted as a kind of “Kavaliersdelikt”, an excusable offense, or even worse, a “Kunstfehler”, a slip of the hand.
True enough, use of SATs wouldn´t make an analyst put skin in the game. What it does, though, is make me sleep better at night as SATs increase the accuracy of my judgements and correspondingly reduce the probability of an analytic error. So, for me, to SAT or not to SAT, that is not the question.
6# What is your approach to intelligence analysis, and how do you apply Structured Analytic Techniques? Are they the key set of methodologies or they are useful but not necessary all the time?
My experience in the use of SATs is mostly limited to analysis of issues and actors in foreign politics. Basically, it´s about threat actor ranking, futures work and stakeholder utility analysis. One thing I absolutely love about SATs is that they allow and stimulate me to create my own data that is based on evidence gleaned from secondary sources and personal interviews. In my hierarchy of life´s delights, creating my own data is the next best thing after ice cream! Even using as few as 200 data points will add probity to my analysis. Recently, I´ve worked with a data set that included 18,564 unique values. And the largest data set that I had an opportunity to handle so far included 33,383 unique values. Relying your judgement on intuition may be worth it, at least for some. But I feel that relying on data is infinitely better.
True, not all SATs are based on data and it´s good so. For example, High-Impact Low-Probability analysis involves developing inferences following a seven-step logical progression. But it still demands structured reasoning that would not allow an analyst to cut corners.
Another big advantage associated with SAT use is reporting. Policy-level decision takers have an attention span of about five minutes. That´s how much time they can afford to spend on reading a particular analytical report, digesting it and taking a decision on it. Which means, no report should be longer than two pages, max, generously spaced and using a 12-size font. Fitting it into one page is infinitely better. For SAT-generated reporting, that´s a piece of cake. Using the inverted pyramid technique, a report will start with a succinct answer to the intelligence question contained in the tasking. Subsequent paragraphs will contain information of progressively decreasing importance. So, all really important stuff will be found on page one.
Oh, I absolutely enjoy teaching this report-writing technique to graduate students accustomed to long-winded and muddled discursive writing. The idea behind it is, only structured reasoning produces clear and concise writing. Once they´ve gotten the hang of it, they won´t do it any other way. It´s so liberating.
7# In the debate around Structured Analytic Techniques, it is often said that they are not manageable when time constraints are too strong. Is time a factor against Structured Analytic Techniques?
Many SATs are based on data. Acquiring or creating data can be a time-consuming process. But if the data is there, you can apply SATs in a flash. Or looking at it from a different angle, the amount of data you will collect will depend on the amount of time available. True, time constraints may affect the accuracy of a SAT-based analytical judgement. But the same will equally apply to judgements based on intuitive reasoning, won´t it?
On my most recent course, exam tasking involved using a combination of long matrices to identify the top-ranking threat actor among seven that figured on the FBI´s Most Wanted List. Students were innocent of any previous knowledge of the subject and had 130 minutes to complete the task including evidence collection from open sources. Despite being based on broadly varying information collected under significant time pressure, students´ judgments showed a remarkable degree of overall consistency.
And even when facing extreme time pressure, you can still conduct pair ranking of seven options in under 60 seconds. That´s of course the most basic of structured techniques. But you know something? In the real world it´s only the simplest things that reliably work every time, all the time. And the result of pair ranking is still ways more accurate and reliable than one produced by intuition.
8# What would you suggest to a student or an early career intelligence analyst? How should they approach the discipline and the profession?
Everyone´s path is different and I´m not the one to say that any one is better than the rest. I can only recommend reading “Psychology of Intelligence Analysis” by Richards Heuer Jr. as well as all books by Nassim Nicholas Taleb. And join IAFIE. Of course.
9# How can our readers follow you?
www.analytic-education.com IAFIE Members Listserv
10# Five keywords that represent you?
Humbleness, curiosity, respect, interested listener.
Be First to Comment