The Butler Did It Again

(This is a follow-up to a series of blog posts on the subject, the most recent being here.) As I was reading another article on the origin of the phrase, “the butler did it,” my attention was drawn to the story The Strange Case of Mr. Challoner, by Herbert Jenkins. Published in 1921, it preceded The Door by nine years. (Interestingly, Herbert Jenkins owned the publishing house which published P.G. Wodehouse’s books, most famously the stories of Jeeves and Wooster.) I tracked down a copy and read it. (There’s a free ebook version of the book Malcolm Sage, Detective on kindle, which collects all of Jenkins detective stories—if you want to read it I suggest you do it now because there will be spoilers below).

Jenkins’ detective was Malcolm Sage, who was at least vaguely in the mold of Sherlock Holmes and Hercule Poirot, by which I mean that he was both very observant of physical details and very eccentric. All of  the stories about Malcolm Sage were short stories, which is very significant to understanding the relationship of this story to the phrase, “the butler did it”.

Novels and short stories are very different things in any genre, but this is especially true of murder mysteries. Novels tend to focus on the unraveling of intertwining mysteries, which is to say the elimination of red herrings. This is somewhat necessitated by the length of a novel; each red herring forms a sort of sub-mystery, which allows one to enjoy the solving of mysteries over and over throughout the course of a novel. There are exceptions, of course. It is possible to combine a mystery with some other genre where the other genre takes up most of the page count. Adventure is the obvious example; a mystery/adventure works well where each clue is the reward at the end of an adventure. To some degree the Hardy Boys books were like this, and to a lesser extent this is often true of the Cadfael stories. The Virgin in the Ice and The Summer of the Danes are both great examples of where the adventure takes up more pages than the mystery. (Both are excellent novels.)

For related reasons—though there are notable exceptions—murder mystery novels don’t tend to focus on figuring out a single ingenious mechanism for concealing the murder(er) for which the evidence was present at the crime scene. By contrast, this is extremely common in short stories. Among other things, they don’t have the space for disentangling red herrings. Short stories which were printed in magazines tended to be extremely short, sometimes only a few thousand words. It also is simply the right size for that sort of game.

The Strange Case of Mr. Challoner is a locked-room mystery. There is one obvious suspect: a nephew of whose impending marriage the deceased disapproves and who will be disinherited on the morrow. The butler was the last to see the deceased alive, and the body was discovered in the library, with all of the doors and windows locked from within. The deceased was staged to look like suicide, and the local police take it at face value. Malcolm Sage makes numerous measurements and observations, and also directs that the photographer attached to his detective agency take a number of photographs. Malcolm Sage is so fond of photographs as evidence that he gives a lecture on their importance to the local police detective inspector. Eventually he reveals that the butler, who had only been working in his position for six months and was highly praised for the excellence of his work, is the culprit. Sage had taken supposedly exclusionary fingerprints from everyone, and used those to find out that the butler had a criminal record and was still wanted. Further, he explained that the butler had put a small metal rod through the hole in the key’s handle and using a string attached to it turned the lock by pulling on the string with the door closed. Once the key turned far enough, the metal rod fell out of the hole in the key’s handle, and he used the string to pull the rod under the door and retrieve it.

Unlike the butler in The Door, this time at least the butler was actually taking advantage of his role as butler in committing the murder. His master didn’t think anything about his coming from behind because it’s the sort of thing that butlers do, and moreover he had an excuse for being in the house after the rest of the household had gone to sleep because he lived there. So at least in this case butling was relevant to the butler’s commission of the crime.

None of the articles I’ve seen so far have cited The Strange Case of Mr. Challoner as having had any influence on the phrase, but then again none of them have cited any evidence for why The Door did have influence, either. It leaves me wondering whether any of this is actually relevant to the phrase I’ve been considering. It might well not be. With murder mysteries having been quite popular ever since Sherlock Holmes first studied scarlet, I assume that there were a great many short stories in the weekly and monthly publications of the early 1900s which have largely been lost to the sands of time. In the days before television and even before radio plays were particularly popular, theatrical plays were quite popular. Wherever there is a maw gaping for novelty, there will be people trying to fill it. Certainly this is the source that the character Broadway cited as his authority that all murders were committed by butlers in the 1933 short story, What, No Butler? I’m disinclined to think that much of the source was movies, though I don’t have any hard evidence for that. Murder mysteries don’t lend themselves well to silent films, though I have no doubt that somebody tried it at least once. The Jazz Singer was the first talkie, in 1927. Talkies took over quite quickly, as I gather, dominating film no later than the mid-1930s and probably in the early 1930s, but that’s rather close to when What, No Butler? was written to have embedded itself in the culture as a common trope by then.

I’m left where I was before, wondering where this trope came from. Perhaps I’ll be successful in tracking down contemporary reviews of The Door, which might be illuminating, but unfortunately a quick google search didn’t turn up anything. I might have to resort to going to the library!

So, The Butler Did It

I’ve been reading Mary Roberts Rinehart’s murder mystery The Door, which I talked about here and here, at five and twenty two chapters in, respectively. This was started off by my wondering about the phrase, “the butler did it”. I’ve finally finished the book, so this post will finish off my review of The Door, and also discuss the idea of the butler being the murderer. I’d warn you about spoilers, but, well, I think that you already know that the butler did it. I might spoil a few side-mysteries too, though, so caveat lector.

The book was in its entirety written in the style of the memoirs of someone who observed a very strange situation. I am used to murder mysteries and detective fiction being, roughly, synonyms, but The Door is very clearly a murder mystery while it is not at all detective fiction. There is a police detective—who does solve the case—but almost entirely outside of the narrative. Several members of the family play at a little detecting, but only occasionally. Only one of them does anything which does not simply anticipate a later discovery, and that was to effect a useful introduction, rather than any actual detection.

The story also maintains the style of foreshadowing hints until the end, abandoning it only as the police detective explains the solution, which is the last thing that happens in the book. I’ve concluded that I don’t like this style. It feels at best overwrought, and at worst like an attempt to spice up a dull narrative with chopped up bits of other parts of the same narrative. I don’t mean that all foreshadowing is bad, of course, but The Door seemed to use foreshadowing in place of a compelling plot.

There is also the very strange question of the narrator, Elizabeth Jane Bell, who narrates the story in a very personal way. Throughout the story alternately laments the tragedy, investigates it, and destroys evidence to try to protect the family. It’s that last part which is especially hard to reconcile with the narration; why on earth would she be narrating all of these scandalous details in a memoir when the character of herself within the memoirs would want all such scandal wiped out? Whether you take the inconsistency between herself in the story and herself as narrator to be a problem with the character or a problem with the narrator (I took it as the former), it is still an unsettling problem.

There is also the problem of the family which Elizabeth Jane was trying to protect. Her niece Judy was never really under any suspicion having, as I recall, an alibi from the beginning. She was the only really sympathetic member of the whole family other than Elizabeth Jane herself, and she mostly from a general pleasantness which seemed to be a combination of decent manners, comfortable circumstances, and little ambition. The rest were detestable. Towards the end I was hoping that the murder would be solved after the good-for-nothing Jim was executed, just so the wretch would be out of the story. The other characters were similarly unpleasant, which left me very unsympathetic to the family’s desire to avoid scandal, which was to a fair degree their only major motivation in anything that they did. But this brings up an interesting point in murder mysteries in general: it’s hard for likable characters to be suspects.

The mystery in a murder mystery obviously depends on there being more than one suspect. More properly, on there being more than one credible suspect. The problem is that a character can fail to be credible as a suspect by being too likable. It’s very difficult to write an enjoyable story about a good person who stoops to murder but then cheerfully covers it up. It’s that much harder to write several characters who are all credible in that way; to pull it off one must write good characters with depth, rather than the common approach of paper-thin automatons who are good merely because they’re not tempted by ordinary temptations. It’s much easier to make suspects credible by simply making there be nothing to which they won’t do for gain.

Another important distinction between suspects in a mystery is between those with an obvious motive and those without an obvious motive. Very often this does not line up well with the moral probity of the characters. In order to put an innocent person in peril (to heighten the tension) a morally upright person will get an obvious motive, while a moral degenerate will get none. This helps to spread the doubtfulness around, to be sure, but because both of these suspects have something obviously going for them as suspects, it is especially common to make the culprit someone who is not very morally offensive (apart from their murders) who has a hidden motive. Which brings us to the butler.

How much was the butler a character and therefore a potential suspect? It’s hard for me to say fairly because I already knew that he did it, of course, but doing my best to be fair, I would say somewhat, but not much. Joseph (the butler) gets progressively more tired, worn out, and on edge as the story progresses, which certainly was a clue (that he was running around doing things while everyone else was asleep). He had originally come from one of the victim’s household’s, which should have been a clue but actually wasn’t—his prior connection to the rich victim had no significance as far that was revealed in the story. Nothing was ever made of him having the opportunity for the murders, because they happened at times when everyone had opportunity, and the house was small enough that a butler’s ability to be unnoticed had no significance. In fact, all three murders happened outside of the house, so his position as butler was—if anything—a disadvantage. He had to sneak off to commit them, or commit them while he was off-duty. The one time his being a butler was an advantage was when he answered the door when one of the victims came to see Elizabeth Jane but he turned her away because Elizabeth Jane was sleeping. Any butler might have turned her away, and any murderer might have learned of her coming and consequently resolved to kill her before she could tell what she knew.

On balance, the disadvantages of Joseph’s being a butler far outweighing the advantages makes Joseph’s being a butler fairly irrelevant to his being a murderer. It’s really just his profession. Most murderers have a day-job and there’s no particular reason it shouldn’t be butling. In this case his being the butler of the narrator was something of a camouflage; it meant that she didn’t notice him. Also his many years of loyal service made her affectionate of him, and this combined with the murders happening nowhere he was supposed to be and her always thinking of him as having no existence past being her butler disguised him as a suspect. But it didn’t disguise him totally. One of the themes of the book is how little one really knows of the people one thinks one knows, and the fact that Joseph had a wife somewhere but Elizabeth Jane had no idea where does actually highlight this blindness in a way that makes it fair game for the reader to not be so blind. In fact, I would argue that line by Jane Elizabeth is a well crafted notice to the reader that Joseph is a potential suspect.

Further, if the test of victory in the contest between the reader and the writer of a murder mystery is that the writer wins if the reader doesn’t guess who the murderer is but blames himself rather than the writer for it, then I believe that The Door has the potential for victory. Reading it through while knowing what to look for, I think that Rinehart did play fair with the reader. Certainly it seems possible she knew who the murderer was from the first, and did not merely cast about for someone she hadn’t already ruled out when she came to the ending. So I don’t think that there’s any cogent criticism to be made of her choice of murderer. (Except, perhaps, that it’s a little odd for someone who engages in fraud, forgery, and conspiracy—which eventually leads to multiple murders to cover those up—to have no criminal history, but instead a long and unmarred career in positions of significant trust.)

So when we come to the question of whether it is legitimate that, as Wikipedia puts it (as of the time of this writing), “Rinehart is considered the source of the phrase “The butler did it” from her novel The Door (1930), although the novel does not use the exact phrase.” Not only does the novel not use that exact phrase, it doesn’t use any even somewhat similar phrase. I’m going to quote the reveal in the novel, but I need to mention a little context first. Joseph had been mysteriously shot in the collar bone about a week before, but he was not killed and recovered enough to come back to his duties, though with his arm in a sling. Elizabeth Jane had, therefore, given him leave to go on holiday to recover. We have not learned up to this point who Joseph’s wife is, but we can mostly guess it was a woman who figured into the plot somewhere else, who we knew to be dying of inoperable cancer. We’re picking up with the tail-end of the explanation given privately to Elizabeth Jane by the police detective. During the explanation he had been calling the murderer “James C. Norton”, which he told her was the pseudonym the murderer had used to procure a safe deposit box. So, with that said, here is the reveal in the novel:

“So we got him. We’d had his house surrounded, and he hadn’t a chance. He walked out of that house tonight in a driving storm, and got into a car, the same car he had been using all along; the car he used to visit Howard Somers and the car in which he had carried Florence Gunther to her death, under pretext of bringing her here to you.

“But he was too quick for us, Miss Bell. That’s why I say I bungled the job. He had some cyanide ready. He looked at the car, saw the men in and around it, said, “Well Gentlemen, I see I am not to have my holiday—”

“Holiday! You’re not telling me—”

“Quietly, Miss Bell! Why should you be grieved or shocked? What pity have you for this monster, whose very wife crawled out of her deathbed to end his wickedness?”

“He is dead?”

“Yes,” he said, “Joseph Holmes is dead.”

And with that I believe that I fainted. [that’s the last line in the book]

There is nothing there remotely similar to the exact phrase, “the butler did it.” As you can see, there was nothing there even related to him being a butler. There were a few things which happened in the house that his living in the house enabled, but much of the criminal activity actually in the house was not in fact Joseph’s doing. The door referred to in the title was a hotel door where a fraud was performed, and was not in the house in which Joseph was a butler. It was not even in the same city as the house in which Joseph buttled. Except possibly as a violation of the tacit convention that the butler is the one person who never, ever commits the murder(s) in a murder mystery, his being a butler is utterly irrelevant either to the murders or to whether one suspects him of those murders.

After a bit of research, I found what seems like evidence that Damon Runyon’s What, No Butler? was first published in Collier’s Weekly, August 5th, 1933. That is not so early that the joke that the butler always does it was necessarily common by the time that The Door was published, three years earlier, but I think it does suggest it. Given what the book actually is, and the timing of it relative to jokes about the butler always being the culprit, I really doubt that The Door was in any way the origin of the phrase. It’s not impossible, but I’d really like to see better evidence for it besides this being the first (and nearly only) book which anyone can find in which a butler actually did it.

 

The Butler Did It?

By an unimportant series of coincidences, I was looking up the origins of the phrase “The butler did it.” The top two relevant results I got were for a trope on tvtropes.com and an article on Mental Floss. The tvtropes article links a Straight Dope on the same subject. All three note that examples of a murder mystery in which the butler was the murderer are rare, but what’s curious is that all three mention a list of rules for murder fiction which SS Van Dine (the pen name of the author who wrote the Philo Vance mysteries) wrote for American Magazine. Though I do have a sneaking suspicion that the two more recent ones may be based on the Straight Dope answer, it is odd that all three cite these rules of detective fiction as if they are authoritative either to what makes a good detective story or to what common tastes were.

Murder Mysteries have been popular for more than a hundred years now, and the idea that there are rules that everyone follows, or that all fans of the genre follows, is absurd. There have been commonalities to detective fiction, to be sure. Giving the readers enough clues to figure out who did it is very common, and very popular, but by no means universal among enjoyable detective fiction. Paranormal, supernatural, and other sorts of detective fiction have been popular. Solutions which could not possibly have been guessed by the reader can be enjoyable as the gradual revealing of an answer. I don’t tend to go for those myself, but pretending that one author’s preference in the 1920s is somehow normative doesn’t accomplish anything.

Within the context of mysteries which aim to be solvable by the reader, most rules (such as Knox’s 10 commandments) aim to give guidance to mystery writers for thinking about the construction of their mysteries. The rules are not meant in an absolute sense, but rather to give sign posts where extra thought is probably required. If the butler, rather than one of the guests, is the murderer, the writer will need to include him as a character enough that the reader thinks that it’s within the spirit of the story to consider the butler.

Now, some might object that it is snobbish to think that the butler is not a possible suspect because he’s just a servant, and indeed it would be, but all problems come with unstated rules, and solving them relies on knowing what these unstated rules are. Consider the classic illustration for teaching people to think outside of the box: Four dots, arranged like the corners of a square, with the instructions to “connect these four dots using only three straight lines without lifting your pen, ending where you started”. The classic solution is to use three lines forming a right triangle where one side goes through two vertices and the other two sides go through one vertex each. This is supposed to surprise people and teach them to “think outside the box” because the rules never said that the end of the lines have to be on one of the four dots. “Don’t limit yourself!” The self-help guru says cheerfully.

The problem with this conclusion is that these sorts of problems are trivial if we’re not helping the person who stated the problem by figuring out what the rules they didn’t state are. No thought would be involved if I just picked up a paint brush and connected all four lines with one thick line. I could even hold my pen against the paper the whole time. Some versions of this mention to not fold the paper; but I haven’t see any rules against cutting and taping the paper. The rules never specified a euclidean geometry; one could easily draw a square then define a geometry in which there were only three straight lines. One could draw new dots and point out that the rules did specify which four dots were the four it was talking about. I could draw three unconnected lines with a pencil while never lifting a pen. etc.

The people who hold this question up as a major revelation are actually practicing a cheap parlor trick. They are really just asking you to try to read their mind and magically know which implied rule they are suspending without telling you. If you were to draw three straight lines plus one curved line, they would balk, rather than applauding you for your willingness to think outside the box in the way that they wanted you to.

The same problem can apply to the butler as the culprit. It would be too easy to assume that the servants are off limits as suspects simply because they all have the opportunity to commit the murder without being noticed, and since detective fiction so often focuses so heavily on alibis, figuring out who had the opportunity is often a large part of the puzzle. Hence this complaint in the tvtropes article:

The butler is the avatar of the most unlikely suspect that, of course, turns out to be guilty because the author wasn’t creative enough to come up with a better way to surprise the reader.

This is a problem only if the butler is the least likely suspect because no time was spent on the butler. Authors who don’t figure out the mystery ahead of the detective, and so who come to the reveal and then have to solve the puzzle for themselves, as it was written so far in order to come up with the ending can run into this. The butler is a good candidate both because he would be surprising since he wasn’t a real character up to this point, and because the servants all have means and opportunity for murder in a great house. This is cheating according to the rules the author implied; to do a good job making the butler the culprit, the author would have had to include the butler as a character in a way that made it clear he wasn’t off limits.

I suspect that this is primarily a problem in mysteries where the author doesn’t know who the culprit is, because it’s all too easy as the evidence is being discovered and alibis are being produced to have accidentally ruled out all of the actual suspects by the end. If that happens, the author will need to introduce a previous non-character who hasn’t been ruled out simply because the author hadn’t thought of the character as a suspect before. I can’t see how such a story can be well crafted; if the author doesn’t know what’s going on, it seems far too likely the story will be inconsistent and not hang together well, though for any technique there is probably someone who can pull it off decently.

But for an example of art criticism which simply wants there to be rules in order to make the task of art criticism easier, consider this from the Mental Floss article:

While The Door was a hit for Rinehart and her sons, who released it through a publishing house they’d just started up, her pinning the crime on the butler has gone down in history as a serious misstep…That The Door was a commercial success while flaunting a hallmark of what some considered lousy mystery writing made it an easy target for jokes. Stories and books like “What, No Butler?” and The Butler Did It soon turned murderous manservants into shorthand for a cheap ending.

Of course this attempt to invoke normative rules of fiction makes heavy use of the passive voice. “Has gone down in history as a serious misstep,” and “flaunting a hallmark of what some considered lousy mystery writing” buys authority with anonymity. There are indeed things which do not need to be attributed—that people will talk about the weather in default of another topic in common does not need to be established with evidence—but common opinion of literary techniques certainly doesn’t fall into that category.

This attempt to have rules of fiction, or more properly rules of art criticism, is not really about the fiction. It is about the desire for stability and intelligibility by a person not willing to do the work of understanding, or without the courage of owning up to their own prejudices and so attempting to displace those preferences onto everyone else.

Incidentally, I looked up the two works cited. “What, No Butler?” seems to be a short story by Damon Runyon. I can’t find much information about it; according to Wikipedia it was in a book called Runyon on Broadway. It was performed on radio in 1946 and that performance is available on youtube. I don’t know when it was originally published. The story does have humor in it, but to call it satire seems like quite a stretch. Early in the story, the character Broadway (who I believe is a theater critic) says authoritatively upon finding out that a man was murdered that the butler did it. When he’s told that the victim didn’t have a butler, he insists that they have to find the butler, because in every play he sees with a murder in it, the butler did it. No one pays attention and he is dismissed because this is stupid advice. In the end we learn that the murderer was a neighbor of the victim, who heard that the victim was rich and so he broke in to the apartment with a duplicated key and killed the victim when he was caught in the act. When asked why he would stoop to robbery, he explained that he was out of work and wasn’t likely to get it again soon. He had served some of the best families in New York, and couldn’t accept just any old employer, because he was an excellent butler. Very clearly, in context, this was not a criticism of the butler as a culprit, but playing with the audience’s expectations to set up a joke.

In 1957 P.G. Wodehouse published a book called Something Fishy. When Simon & Schuster published it in America they used the title,  The Butler Did It. Wikipedia gave this plot summary:

The plot concerns a tontine formed by a group of wealthy men weeks before the 1929 stock market crash, and a butler named Keggs who, having overheard the planning of the scheme, years later decides to try to make money out of his knowledge.

(Tontines are in themselves an interesting read. It’s easy to see why they would show up frequently in older detective literature.)

According to the further description of the plot, Keggs is long retired by the time the book takes place. His being a butler is incidental to the story, so far as I can tell, and doesn’t seem like it can be taken as any sort of criticism of detective fiction where the butler is the murderer. This seems doubly true given that The Butler Did It was not the original title, and was only changed because it would resonate better with Americans.

And now that I mention that, it occurs to me that all of the discussion of butlers, from Rhinehart’s story to the supposed criticism of it is all American. Aside from Poe’s character of Dupin starting the genre of detective fiction, much of the most influential detective fiction is British. Now I wonder whether “the butler did it” is a primarily American phenomenon. In any event it does seem to be a very curious example of a saying without much basis, used at least as often to joke about the saying as even to say anything about detective stories.

If I had to guess, I suspect that it originated with someone who was complaining that detective fiction is very formulaic. If so, it is ironic that they picked to exemplify this putative formula a feature which is extremely uncommon in detective fiction.

Having said that, it occurs to me that this idea could even have originated to mean nearly the opposite. It could have started as a parody of the sort of person who doesn’t know how detective fiction goes, and who leaps to the butler as the obvious suspect because he had the means an opportunity for the murder. It would make a more effective criticism of a naive reader than of murder mysteries. “Pffh. He’s the sort of guy who decides ten pages in that the butler did it!” As it stands, I see no more evidence for any other theory of where the phrase came from.

 

Intelligent Murder Mystery Suspects

I’ve recently watched the episodes in the thirteenth and final series of Agatha Christie’s Poirot, starring David Suchet. It included Curtain, which of course must be the last episode, but it had several episodes which differed very greatly from their source material. In particular, The Big Four and The Labours of Hercules.

The former was described by the screenwriter as an unadaptable mess, which it certainly seems to be looking at the plot summary. It is basically a spy thriller with dozens of characters set throughout Europe, which is not very viable for a TV show, even if it is nearly two hours long. The one which really interests me, though, is The Labours of Hercules. The original is a collection of twelve unrelated short stories, which the screenwriter turned into a single long-form mystery by taking one of the stories as the central one and using several of the other stories as the red herrings which one expects in a Christie novel. At this point, I should warn you that this post will include spoilers. You have been warned.

Given what a challenging prospect that is, the writer did a good job, but there were problems in the story which I do not think were avoidable for structural reasons. As everyone knows, a murder mystery must have suspects, with the plural being imperative. Every man having free will, anyone who was anywhere near the victim is a suspect, which is why an isolated setting—a mansion, a private island, etc.—is so interesting. Unless the author is cheating, the suspect list is known at the outset. When doing this, the author must be very careful to make all of the suspects believable suspects. That’s a universal criteria, but a murder in the middle of a city means that we see a great deal less of the suspects, so each one has far greater scope for unseen action, including accomplices we don’t know about yet, than people in an isolated setting.

The episode, The Labours of Hercules, was set in a hotel on the top of a mountain in Switzerland, with the funicular train that is their only link to the outside world having been shut down by an avalanche. Short of a ship in the middle of the Atlantic ocean or an aeroplane in the sky, it’s about as isolated as it is possible to get.

The central mystery, though Poirot stumbles onto it almost by accident, is the identity of a psychopathic killer and thief called Marrascaud. The mystery was set up in the beginning where Marrascaud managed to kill several people and steal several valuable items—one of them a large painting—from a crowded building, with disguised policemen and Poirot himself protecting them. From this we know that Marrascaud is a genius on a level with Poirot, and this forms the central problem once we get to the hotel.

As has been observed in countless murder mysteries, the best place to hide a leaf is in a forest; to hide a genius one must really put them amongst other geniuses, but the characters at the hotel were taken from other stories and thus had qualities appropriate to those stories, none of which involved unique genius. In this case, the beautiful daughter of Poirot’s former love interest who is fascinated with criminology stands out almost like a sore thumb; the only other person who comes close is the Countess Rossakoff, her mother, but it was very clearly established in the previous episode where we met the Countess that the character is not a murderer. Marrascaud kills for the pleasure of it, brutally, which is not something one degenerates to in old age. It is true that one can be cruel vicariously, through underlings, in old age, but it does not make sense as a personality change to go from an honorable thief to a psychopath who delights in killing.

An interest in criminology is also something of a red flag in a suspect. Though everything has by this time been used as a false flag in detective fiction, none the less the similarity of the violent nature of both crime and law enforcement is unavoidable. As the saying goes, the main thing which distinguishes a sheep dog from a wolf is who it bites. None of the other guests seemed sufficiently… canine.

I think that this is the reason why Conan Doyle put Moriarty as the mastermind, behind the scenes. The proxy of an evil genius need only be of ordinary intelligence, which makes it far easier for him to blend in. Indeed, executing a plan which requires greater intelligence than he himself possesses serves as a form of camouflage for the immediate villain. Still, as bumbling accomplices have long shown, it is best to choose someone intelligent enough to understand the plan once it has been created; an accomplice who can understand only his part and not what it fits into will make mistakes that will prove the undoing of both.

I think that fact is why some villains have tried to manipulate their accomplice into helping without realizing it; if done well the mistakes of the unwitting accomplice actually hide the involvement of the mastermind. I suspect that this is the ideal strategy for the criminal mastermind; it is the safest type of plan if a brilliant detective shows up. If done extremely skillfully, it is possible to conceal that there even is a brilliant plan at work; the brilliance can be disguised as coincidence.

Of course, mysteries can go the other way—the more realistic way—where the detective must make sense of genuine coincidences. The problem with writing this sort of mystery is that it is extremely difficult to pull off without the detective himself getting lucky. And while a comedic detective—Inspector Clouseau, for example, or taking the idea of a detective very loosely, Maxwell Smart—can stumble onto all of his solutions, it’s not entertaining if a serious detective does that. Though, I should mention that this is why Jessica Fletcher almost invariably figures out the solution of most episodes by chance. In order to make Murder, She Wrote accessible to a general audience, the writers would tend to throw in enough clues that one should be able to figure out the solution before Jessica does. Since Jessica does have to figure out the mystery, something must make her realize the solution, and because we the audience are supposed to already get it, it can’t be the last piece of critical evidence, but nor can it be slam-dunk evidence, because then you couldn’t feel smart during the reveal. So it’s usually something silly somebody says, and then Jessica says, “Wait, say that again? Of course! That’s it!” That’s not literally every episode, but it is basically a structural requirement imposed by the show’s relationship with its audience.

The solution to the mystery depending on figuring out coincidence without the detective merely getting lucky is typically easiest to pull off through exhaustive leg-work—checking every chemist’s shop in a 30 mile radius, that sort of thing. This is why that sort of mystery is most common when the detective is a public detective (i.e. a member of the police) rather than a private detective, or at least when the police and the detective are working together, rather than separately. And even then, Sherlock Holmes had his Baker Street irregulars.

The other approach, which is a compromise that keeps things closer to a  detective the reader can relate to, is for the detective to have something to go upon which through intelligence, knowledge, and wisdom allows him to rank coincidental possibilities according to an order they are likely to have happened, and to be right according to a Poisson distribution (basically, they usually get an answer by their third try to verify a coincidence, sometimes it takes a lot of tries, and because no one has infinite effort to give, sometimes they don’t get an answer). Fundamentally this is still the detective getting lucky, but it is a way for the detective to earn his luck. Since the detective doesn’t create the clues but only discovers them, that’s the best he can do in any case.