The most amusing variant of the XY problem is when you want a seemingly ridiculous Y, and then, when people inevitably ask why, you tell them about your entirely-novel X, which they still don’t understand the use for, so you have to explain the entirely-novel W that motivates X, and so on... and by the time you’re done, you’ve explained your whole business model, written a ten-page journal paper, and completely killed the conversation—because they don’t feel they have the prerequisite knowledge to talk about A-through-X, so you just get a “good luck with that, buddy.”
Even though Y was a concrete problem that they do have domain experience in solving!
I call this “going up the rabbit hole.”
——
Example (deliberately shortened from its real depth):
• I’m trying to invent a novel algorithm for inter-procedural data-flow analysis, because none of the existing ones work for me.
• Why? Because I’m trying to reverse a stack machine bytecode with only indirect jumps and no reified CALL/RET opcodes, into a structured inter-procedural control-flow-graph.
• Why? Because I want to do program-slicing to recover HLL representations of key-value-store writes that the program does.
• Why? Because I want to use them as hints for my runtime execution tracer for said bytecode, so it can emit better traces.
• Why? Because I want to put those traces into an OLAP database and I want the data in the DB to be typed.
• Why? Because these traces are the only canonical representation of the state of the system, and our business model is allowing auditors to figure out what’s going on with the system.
This is why on the rare occasion that I find myself answering questions on IRC or Stack Overflow I always start with at least a simple answer to the question asked. Only once I've taken a stab at answering the question do I consider moving on to "but, what are you really doing..."
Another related type of question is a question born out of curiosity rather than need. Sometimes you want to do things just for fun to see how they work, but you get shut down every time you ask a question because "you should never do that in production".
I think it's generally good to always do both, on both sides. If you're asking, start with what you want, and then a why. If you're answering, I'm not sure the order is as important, but starting with whichever is shorter is probably a safe general rule.
The asker knows that providing the full A-X context would be an utter waste of time, and really does want an answer to Y only.
Certain IRC regs will flat out refuse to answer Y. Often this is because they don't know the answer to Y, but instead of saying so would prefer to wear the mantle of wise interrogator of "things you did not think of"™.
Ofc, sometimes they're talking to a newb and this is appropriate, but jumping to "this is an XY problem" seems to be a knee-jerk reaction for some people.
One of the more effective albeit mildly dishonest to get around this on IRC is to troll the channel.
“I asked my buddy who claims to be a language C expert and he told me that it is completely impossible to accomplish Y using language C, because «invalid reason Q that makes language C sound bad».
Then all of the smug jerks who would usually start by asking what kind of idiot would ever want to do Y will instead fall over themselves to prove that Y is possible and really quite simple to accomplish, and explaining that my expert friend who said it was impossible had no idea what he was talking about.
Make sure to use a pseudonym for this one though if you want to come back to the channel and not have people still mad at you.
I hate that, I ask for help with an odd Y and say "I've researched all other solutions and they didn't work, so can someone please help with Y?" but people still make me show them my work and convince them that everything else actually won't work before they maybe help.
In my experience this can happen for two distinct reasons (not exclusively):
1. The StackOverflow Reflex, where the person responding doesn't believe you've actually explored other solutions sufficiently because in their experience the vast majority of the time the person asking for help hasn't actually explored other solutions effectively. This may feel like bad faith (from either side), but it could just be the asker's inexperience with other solutions that cause them to not apply them correctly.
2. They don't know how to do Y, but they know how to do X really well so they want to make sure you haven't missed anything in your attempt to make X work. Remember that different people have different expertise and may want to try to help solve your problem even if they don't know how to solve it with your chosen solution.
I think askers often don't recognize that explaining why T-X won't work in their case is actually very helpful to fully understand the problem, even if the person helping ends up solving Y directly. If you're asking for help solving a complex problem, sometimes you have to teach the problem to the person helping you. And that's ok.
>The StackOverflow Reflex, where the person responding doesn't believe you've actually explored other solutions sufficiently because in their experience the vast majority of the time the person asking for help hasn't actually explored other solutions effectively.
Heh. Encountered this just the other day. I asked a question about how come an ugly kludge was faster in Python for doing a particular task, and mentioned that I had tested this using timeit and perf.
I got a bunch of answers back that assumed I was doing something stupid that I didn't really need to do. Needless to say, the code samples they posted all ran 5-10 times slower than my kludge, which means they hadn't even tested them.
I agree, and there's no simple way to signal that you really do know your stuff and really have investigated other solutions. The one or two times I tried this, responders became slightly hostile after a bit, implying that I didn't know what I was talking about, that I was doing things wrong, that my constraints weren't real, etc.
It was probably as frustrating for them as it was for me.
I've found that if I need something oddly specific most people wouldn't be able to help anyway (and those who could would have to invest a lot of time in that specific case to understand it).
I've simply stopped asking for help with those. IMO there's some point after which it's counterproductive for both parties.
> I've researched all other solutions and they didn't work,
I ran into this the other day. A person was asking about a Y and when pressed about it that person said the other solutions didn't work. I dug back into that person's history and while they did try a canonical solution - the lacked the ability to debug their efforts.
For context, the Y they wanted to do was essentially to rewrite a core framework in Rails because they didn't understand that an object they were using was nil. Their problem was entirely about how they were getting input from a user form and they wanted to rewrite a framework.
When I pointed it out the person never responded, and I saw a similar question posted by that same person a couple of days later.
I get this sometimes when I'm trying to learn about a suboptimal solution. I can even preface the question with, "I know this isn't the best solution but can you explain why it works..." Inevitably I get, "you shouldn't do it that way..."
Sure, there are cases where seemingly-ridiculous things are actually needed.
Where a lot of people commenting on this article are failing, though is thinking only about their needs and what they want. The people responding to your IRC message/ML post/etc are (usually) giving up their valuable time to help others for free, and making some accommodation for this is essential if you want to get anywhere useful.
On top of this, when you join a room/list where you aren't well known, the responders have no context on who you are, your level of experience, etc[0] - but what they DO have is long, long experience with people asking XY questions where Y is both ridiculous and not going to solve X anyway.
I know it's frustrating to have to go through your reasons when you are certain that damn_silly_thing_Y is actually the only way of solving your problem, but it is needed. After all, you're asking these people because they're likely more experienced than you are (or else why ask them?) and thus there's a chance you could be wrong.
0: And you can't often shortcut this - due to astronomically high rates of Dunning-Kruger cognitive bias in many of these venues, starting out with "I'm an expert, I know what I'm doing" without very convincing evidence is usually extremely counter-productive.
I think you confused my point for the point of the OP, and then just typed a reply that is basically a reply to the OP.
My point was the fact that, at the end of the justification chain, you've generally left the domain of expertise of the expert/forum of experts that you came to. Because of this, the people you're conversing with will fail to actually help you with your problem, because they've been dazed/confused/distracted by the higher-level problem statements that exist outside their domain of expertise. Even though the immediate problem is one they're eminently qualified to solve.
It's unintentional nerd-sniping: they force you to tell them about a larger problem that they don't know how to solve—sometimes, in my case, because it's a research problem and nobody knows how to solve it—and they get distracted trying to understand the entire breadth of a topic they may not have ever considered before, in order to figure out how they would solve the highest-level problem themselves. They never "come down to earth" to solve your immediate problem.
It's as if you went to a low-ranking soldier and asked something about how well some body-armor they use works in practice, and they asked you what the context is, eventually demanding you to explain the whole strategy of the war to them, at which point they get confused trying to understand the strategy and politics of the war they're fighting, which is usually inevitable because they're a soldier, not a politician.
I don't think I am. I can certainly understand that it's doubly frustrating when after all the explanations you still don't get an answer - but there's a couple of further issues with your point specifically. As has been noted elsewhere, questions need to be appropriate for the venue they're asked in. A general-purpose channel for <language>-users may not be the best place to ask extremely niche questions about algorithm design in edge-case scenarios where none of the usual answers apply.
If what you're doing is hyper-specialised to your individual environment, there's a very real chance no-one will know the answer. If you put the effort in to explain, in detail, WHY you need this really strange problem, you may get lucky and someone smart and experienced in a closely related field may be around and willing to put in the time to help you. Or not. But you're still going to need to be willing to to in depth into the reasons behind what you're doing or they will likely ignore you.
The lack of response probably isn't because people are dazed and confused, it's probably because your constraints are so restrictive that what you need isn't free IRC help, but a specialist contractor who you're actually paying for their time.
Do note that I'm not talking about asking questions on IRC; but rather, I'm talking about asking questions of people who I already know to be experts in the specific field I'm asking about. (The "specialized contractors" you're talking about, but more like e.g. university professors.) Like, if I have a concrete graph-theory problem, I'm asking a graph-theory expert about it.
The problem doesn't come because the population I'm asking doesn't know graph theory; it's because someone who's an expert on graph theory can't be expected to also know e.g. DSP design, and the higher-level problem is a DSP design problem, and once they find that out, they say they don't know DSP design and refuse to help. Even though the concrete problem is a graph theory problem!
This is, IMHO, one of the key psychological problems hindering interdisciplinary fields of study: everyone thinks that, because a problem is A&B, and they only know things about A, that they can't make a contribution. Even though everyone only knows either A or B, and the whole point is to get the people who know A talking to the people who know B, rather than to create some sort of A&B paragons.
Often a key problem with interdisciplinary study, as you described it, is that the people who know A and the people who know B are often both bad at communicating outside of their expertise. Sometimes terrible. Even worse sometimes when they teach A or B, because they incorrectly believe they are good communicators due to having some success as lecturers, but don't realize how narrow that is.
So getting them to talk to each other often isn't nearly enough - you need a translator in the conversation, or you need to teach them to be more effective communicators.
I've just started immediately telling them my high level project if they ask why I am doing something. Even though I really doubt it will aid understanding and fixing my problem, since my high level projects are highly obscure research projects, it at least saves time. And then they'll either go away, or give a solution to my original problem; if they start questioning my research project I can just ignore them, though that has not actually happened yet.
What's the basis for this extreme skepticism, is it an accepted law that every problem in programing can be solved with some basic plumbing? My guess for why so many people love the X->Y thing is that it allows them to join in on discussions where they have no idea how to answer the question.
I mean, past that point, you've left "technical problem" land and entered "a description of what the business sees as an exploitable market opportunity."
But to be pithy, it's because the system is all of:
• intentionally brain-damaged for OLAP, by a design focusing on OLTP efficiency to the exclusion of all else (think: bit-packing structs into k-v values in a way opaque to anything other than the program that owns the values; no binary object-wrapper format like ELF to describe which ISA is in use by a bytecode program; etc.)
• very popular (enough that auditors are interested)
• a competitive mutually-untrustworthy multi-tenant platform, where people have every incentive to obfuscate any sort of event logs their programs emit, so that they can keep the competitive advantage of analyzing those logs to themselves
Or, in short: the system is a scrambled egg, and people are willing to pay to have it unscrambled for them. Most developers would just say "you can't unscramble an egg" and give up. But the schema information is there to recover—it's just burned into the logic of bytecode programs. Someone with the right expertise can recover it.
It's pretty lame that the examples on the page show the answerer not just answering but also talking down to the questioner. XY is a problem that happens a lot, but disparaging people who are trying to learn is not the right response.
Here's how that conversation should have gone:
---
Q: How can I echo the last three characters in a filename?
A: If they're in a variable: echo ${foo: -3}
A: However, if what you really want is to get the file extension, it's probably better to handle extensions of any length. Try this instead: ${foo##*.}
A: That expression says to find the longest prefix of `foo` that matches the given glob pattern, and remove it.
Q: Oh wow! That is actually what I wanted, but it hadn't occurred to me that such powerful pattern-matching would be built into the shell. I already had a solution involving piping through sed, but it was pretty slow and ugly. I noticed that all the files I'm operating on have three-letter extensions and I figured there'd be an easier way to take a slice of a string, since that's a common thing to have built-in. But your answer is the best of both worlds. TIL!
---
There's really no need to admonish the questioner for starting with the wrong question. That won't help. It will only make them afraid to ask questions at all.
You proposed interaction assumes that the respondent was able to guess the users ultimate goal. That's often only the case on the Nth run-through of the exact same proxy question.
Asking the wrong question wastes both the asker and the answer's time. By all means be kind, but if you fail to take the opportunity to remind someone to be wary of this general problem you are doing both them and yourself a disservice.
This is a great attitude. I tend to approach junior technical staff asking these sorts of of questions with a slightly different tack -
Q: How can I echo the last three characters in a filename?
A: What is it you're trying to achieve by doing Y? What's the context? You can probably do it this way ... , but let's see if it's a good fit for the problem you're trying to solve.
I think it is worth telling (politely) the questioner that they really should start by telling people what they really want to do, in order to get better feedback in the future.
I think the most polite way to tell them is not to tell them what to do, but rather ask: "Could you give us more background on what you're trying to do? It might turn out we know of an alternative solution that avoids this specific problem."
This doesn't suggest that the questioner did something wrong, but still helps them learn to provide background details when asking questions.
But people don't learn or change behavior if they don't understand the issue. In this case, providing polite and constructive feedback would help them ask better questions going forward.
I will say something like "can we back up a minute? Why are you trying to do this?" And yes, my second question should be worded differently or spoken carefully to avoid being interpreted as "WTF are you doing?"
yes, surely the questioner is trying to balance “doing as much of the work as possible”, and “breaking the problem down into small steps”, versus “give me the answer to my entire problem”
If you ask too broad a question you’re likely to be ignored, imo.
> People are there to voluntarily give help. They also get loads of really stupid questions and venting does help with keeping them providing help.
Ha, imagine hearing this about volunteers at a hospice. SO boards are infinitely less important, but volunteers shouldn't be losing their calm at the same people they're helping
If you must vent, vent privately to a friend. If you can't help but vent back to the person asking the question, then please stop answering questions, because you aren't helping.
If you're a seasoned veteran here to answer my stupid question and the cost of answering my stupid question is to condescend then PLEASE KEEP WALKING. I do not want your help.
I disagree with your assertion that the QA community would be worse off if these types of people stopped answering questions. If a person cannot provide a helpful response to a question without being condescending or rude, then I personally think the QA community would be better off if that person reduces the 'help' they are providing.
Very few people have knowledge that is so rare and difficult to obtain that others should endure discouragement (or plain unpleasantness) in order to receive the benefit of their help. And, in my experience, the people who do have that sort of specialized knowledge are often some of the most helpful.
I agree with the rest of the commenters here that the XY problem problem is a way bigger one. Example:
"How do you implement a new system call in the Linux kernel?"
"Ah, well, a system call is overkill and leads to compatibility issues, you should try to see whether you cannot solve your problem some other way..."
"My problem is I want to understand how system calls are implemented thus I want to go through all the steps of making a new one so that I don't miss something"
Or, a more recent one from Reddit:
"When will we have desktop computers capable of replicating the feat of Google with AlphaGo?"
"Ah, well, we already have distributed computing trained NNs, also there is some research that it may not require such power..."
"I'm not interested in Go, I'm interested in the power of future computers!"
Seriously, people second-guessing you are annoying.
> "I'm not interested in Go, I'm interested in the power of future computers!"
Reminded me of this story. I told a co-worker, while we worked on our DB:
- I wish we had the money to run Oracle.
- Are you crazy? Postgres is awesome and perfect for us!
- I know. I'm not saying "I wish we were running Oracle". I just wanted to have the money to do that. :)
How is the second answer second-guessing? Replicating the feat is a question of both software and hardware. To give an answer that's just about hardware would require second-guessing the intent of the question.
try asking "how can I automate ssh authentication by password?" sometime. I've still never found an answer other than "never do that, here's how to use ssh-agent."
There's actually a pretty good Stackoverflow answer to this one: https://stackoverflow.com/a/43526842 (Which I think is a good pattern: sure, mention the "you probably should consider Y", but show the alternatives as well)
They want to do easily something that is hard. I can understand their revolt because it shouldn't be hard, but it is.
The easiest way out is probably #1, what probably will need the cooperation of some uncooperative 3rd party. It's the correct question, it's just that the answer sucks.
There's another anti-pattern that I can't recall the name of, where instead of asking if something can do something, you make the claim that it can't, and let them refute you. It's excellent for something people get very defensive over, like Unix.
When I tried googling your exact question, the first hit (https://serverfault.com/a/512220) was a SE post where the highest voted (though not accepted) answer was doing exactly that with `sshpass`
Tbqh, without you explicitly stating that you're aware of key pairs and concluded that storing plaintext passwords is the best solution for you, I would excuse anyone who thought that you maybe just hadn't done the research.
> "How do you implement a new system call in the Linux Kernel?"
> My problem is I want to understand how system calls are implemented thus I want to go through all the steps of making a new one so that I don't miss something"
These are very different statements, and require different answers. It's like asking "How do I sort a List" vs. "I would like to understand sorting algorithms, what are some resources to start with?".
People second guessing led to what you actually should have asked in the first place.
That's definitely a case where asking that question wasn't warranted. You asked a question about a factual thing. It's essentially a yes/no answer, not a "how to" answer, which are way more prone to the XY problem.
What happens sometimes is that X is supposed to be simple but it doesn't work or it doesn't work for that specific case, so you're trying to work around it by doing Y (or Z, etc.)
That feeling when you spend hours reducing a problem to its simplest form and then the experts pile in to your IRC / mailing list / ticket system / stack overflow post to explain that you must be suffering from the XY problem because your ultimate goal couldn't possibly be to do what the minimized program does.
Naturally, they didn't read the part at the top where you explain your ultimate goal.
Two days later you make a new post favoring contextual detail over brevity and the same experts pile in to complain that you didn't minimize your example.
I just had to left a job after 2 weeks, because as someone new to the clan, everything I said was noise to them. I guess there's a time-to-respect period to go through.
I don’t know about this particular situation but I think it’s just a matter of politeness that somebody new should use his first weeks to try to understand the current situation before making too many suggestions. Also show your skills by delivering something.
I have seen quite a few contractors who didn’t seem to be able to do anything if the environment was not exactly their preferred environment. If you are good you can deliver something in any environment and build up credibility.
It's partly that yeah, in my case the issue is that what they called skills were just hardship turned into experience which didn't call for their propensity to insult / accuse me (behavior that they rarely did between themselves).
Basically it's clan mentality that makes it hard to integrate because they don't transfer knowledge or confidence but authority and rot learning procedures.
I'm a bit confused by its pictures. It has a scroll wheel, right? How's the build-in wrist pad?
I'm still using a Kensington Expert Mouse.
Anyone tried those vertical mice? To me it looks like a pointing device for people with RSI rather than something one may choose for better usability whether they have RSI or not.
I was introduced to trackballs by a friend who has RSI, and I loved them, even though I don't have any wrist issues.
There's no separate scroll wheel — it scrolls by turning the ball around the z axis, so it's purely optical. It works very well unless you need to keep the x,y absolutely fixed while scrolling. I can't comment on the wrist rest; wrist extension causes me trouble, so I have mine sunken into a customized keyboard tray to place the ball near keycap height.
Unfortunately, the assumption that "someone who's asking for help with something doesn't really know what they want or need " breeds horrible culture.
This is evident by the examples in the article and the ones in the cited sources. If someone is asking for help, don't try to shame them into explaining what "they really want". What they want is help with a morsel of a problem they ask, nothing more. If it's not the right path to a solution, they will decide, not you.
> If it's not the right path to a solution, they will decide, not you.
This is a dangerous position to take. I have a junior dev on my team frequently asking for help on Z when X is still a problem. If I actually answer his question, what he puts out is a disaster.'
Like he was having trouble with generating prime numbers. I could have answered his question, but why is he generating prime numbers? Long story short, to create an "optimal" number of threads to insert to a database because inserts were slow and he had 100k to do. Dude. Batch insert. Why are we threading, absolutely not, no. Stepping back Z, Y, X, and V and finding a better solution. 1/10th the code, 1/20th the time, better for the DB.
When you have someone on your team doing things like this, absolutely ask about X.
It doesn't need to be a negative, condescending thing. Taking the time to ask "What problem are you trying to solve?" can alleviate quite a bit of frustration and wasted time. It's a simple question. It's not assuming they don't really know what they want or shaming them—You're trying to better understand the problem they're solving so you can give them a better, more holistic solution.
This applies to conversations on the Internet in general (and probably to meatspace ones too) - unless there are obvious signs suggesting a different meaning, taking what someone wrote in any other way than at face value is disrespectful and rarely leads anywhere productive.
>shame them into explaining what "they really want".
You're already starting from a toxic situation where asking for help is somehow bad. Besides, you have the same "I know better" attitude in this case, just on the opposite side.
It might get on some nerves to start having to answer "why do you want to do that?" when you're in a siloed culture but eventually people adapt and it just leads to more thought out plans.
Asking why just adds more context and it doesn't need to turn into a second guess of the solution. Its better to breed a culture where asking for context is considered a good thing with no emotional baggage tied to it.
I agree that shaming / confronting people is not good for culture, but I disagree with the conclusion. "Y can be done (see below), but if you're actually trying to accomplish X here's a much more convenient way" is the kind of response that I appreciate most–not only has someone answered my question, but they've also helped correct whatever misunderstanding prompted me to ask about Y in the first place, and they've given me an easier way to solve X.
For me, at least, a response that solves both X and Y is (if done politely) far preferable to one that only solves Y.
Most comments here are about how annoying people asking about X are.
We get it, sometimes you are asking about X simply because you need to know about X and you know what you are doing.
But often, especially when dealing with more junior engineers, I've found this simple to keep in mind and helpful. It's also a good proof of experience to know when to ask about X and try to make sure you understand the context of the question before answering it.
It can be telling how people react to being asked about X. I tend to ignore conversations if someone gets annoyed about getting asked what they're really trying to do. Though you do have to consider whether they just don't understand enough about what they want to do, to know what question to ask.
A special case of Y questions are the overly abstract or general ones. "How do I [solve this kind of problem]?" It's understandable that people try to build up a repertoire of problem solving techniques, but this can be very annoying if the specific X question is simple to answer, whereas the general question is difficult. Sorry to say, but I tend to also ignore questions (I'm thinking of chat rooms) that seem overly general any more, unless I'm in a mentoring mood.
Agreed. And sometimes X is not permitted or unlikely to be permitted.
In my day job, I work at BigCorp and I get questions like "how do I Y without Z?" Often, the answer is "you shouldn't Y without Z. In order to have a consistent UX, all BigCorp applications have a policy that Y is done by Z... What is your X?" And then we can talk about if X really is exceptional.
There is a world of difference between a forum like stack overflow where individuals have diverse needs and a forum inside a perscriptive company where uniformity is an engineering or usability booster.
OTOH, wanting to do y for other reasons, finding an xy SO post on Google, and not being able to get the answer because it went off topic about rather or not y is the solution to X is frustrating.
Bonus points, ask y again because the other question answered z only and get it closed as a dupe of the other question.
I'm one of the people who tend to ask why. Sorry to everyone on this thread who are frustrated by it.
From my point of view, I'm really not trying to undermine what you're doing, it's a genuine attempt at getting you the best solution.
Here are some examples:
>How do I use apt on Fedora?
>How do I unroll this bash loop?
>How do I multiply this number by 1024 with sed?
I could have a lot of fun answering each question exactly as stated and if you say "I know it's the wrong tool for the job but I'm doing it for fun" that's exactly what I'll do.
However, it turned out that:
>They just wanted to install stuff thinking apt was universal. The better solution was using yum
>Their loop was slow and they had read that unrolling was an optimization. The better solution was reducing the number of subshells spawned
>They just wanted a result, not to implement an integer multiplication state machine in sed. The better solution was using perl/awk
I know some people are jerks about it, and that sometimes you want answers to weird questions. When people do appear to be going down the wrong path though, it would be a disservice to help them do that without letting them know that there's probably a better way.
One thing I hate is when the XY problem gets encoded into a knowledge base:
1. It's common for people trying to do Y to ask about X, which is a bad way to accomplish Y.
2. I'm trying to accomplish Z, for which X is a perfectly good option.
3. Every answer on stack overflow for "How to X" is people assuming the XY problem and answering "here's how you do Y instead".
The question, "what are you trying to do?" is super useful to remember to ask. I work at a company that's trying to modernize and move to devops for dev environments. Because the infrastructure is not nailed down, this means that every single team needs to have at least one person who understands the devops.
The process of learning k8s and grokking the security concerns in the course of a few sprints is not pleasant. This "XY Problem" crops up all the time, but the devops team is absolutely slammed all the time so many times basic queries go unanswered. Everybody has to help everybody else.
I don't think anyone wants to go back to the days of rigid development infrastructure so we're kind of stuck with devops until the ecosystem can mature.
My great hope is for serverless to take off and so programmers can further specialize. I'm getting tired of learning new languages and stacks every few months or years. Just hand me a biz requirement and let me implement it any way I like.
Conversely, there is the XY fallacy: when one [frequently condescendingly] assumes they know better than the person asking, errantly classifies it as an XY problem, and fails to answer the original question.
Yes. Sometimes X is a hairy problem and perhaps well beyond the scope of what anyone cares about, and Y is a well thought out distillation of a sub-problem.
One of the problems I have with stackoverflow is crafting a question so that only those who have the potential to answer the question will answer it. Unfortunately there are always the folks who want to be first to answer the question and don't understand the fullness of the question and proceed to give a quick, poorly thought out solution. This then keeps others from chiming in.
For example, in one question I was asking about JS array performance and was using bubble sort as just a way to exercise arrays. Immediately I get folks mocking me for using bubble sort -- they completely missed the point of the question.
False positives on xyproblem appear to be much more rare and much less harmful than true positives.
At the end of the day a good question provides both context and an indication of things that the asker has tried/learned so far. The person being asked is a person, not an answerbot.
Depends. This is a matter of prudential judgement. The proper attitude is contextually determined and the linked page lists some qualifications, though still takes too much of a narrow, absolutist attitude.
For example, if someone random poses a problem on IRC, the solution to the "real" problem to be solved is NOT the concern or responsibility of the people in the channel. Unless there's a reason to ask for more context, just answer the damn question. Blindly asking the user what they're REALLY trying to solve is taking an intrusive and patronizing attitude. However, if asking will provide helpful context, then that's a valid reason to ask. If the problem as posed seems onerous or contrived, you COULD courteously ask the other person what they're ultimately trying to solve in the spirit of offering to be helpful if the other person so desires, but no one is hiring you do solve the "real" problem for them, so mind your own business.
If a coworker approaches you with a question, similar principles apply, but if you're working on the same project and in the same context, it may be natural to ask about what the larger problem is in a spirit of collaboration, esp. when the questions posed by your coworker smell funny. After all, you both share concerns about a project, though again, this does not give you license to automatically wrest the problem from your co-worker. His ticket, his problem. Your role is advisory.
So, common sense. Don't be one of those doltish know-it-all jerks who like to run the show. Mind the proper boundaries of concern.
Is it just me or is the XY Problem really poorly named? It's unlikely that anyone could hear "XY Problem" for the first time and have any idea what it means.
These are mirror-image manifestations of the same underlying problem. In the example given in the article, the person asking mistakenly thinks X and Y are effectively equivalent. In your case, the person replying makes the mistake.
XY problems are just the tip of the iceberg; many problems are the result of mistaken assumptions of equivalence, and they often go unstated.
Here's the thing, if you ask an unusual question, one of two scenarios is often playing out:
In case 1, you are a beginner. You have an XY question. But you've never heard of an XY question. You also probably couldn't identify your question as an XY question if you had heard of it. Trying to educate people about XY problem is pointless, because beginners don't see it or understand it or have the ability to apply it.
In case 2, you are advanced enough to avoid asking XY questions. You are instead stuck on some esoteric question which you can't resolve by exhausting all your skills. So you post a question on the internet. But you are only asking a question because it is a really hard question. Chances are that most other people cannot answer the question either. So, the only responses you get are people suggesting that you have an XY problem.
> Trying to educate people about XY problem is pointless,
I haven't found that to be the case.
Neophytes are perfectly capable of asking questions with a reasonable amount of context: they just need to be taught to do so even when it seems unnecessary to them. Polite explanation to this end works.
> to avoid asking XY questions. You are instead stuck on some esoteric question which you can't resolve by exhausting all your skills. So you post a question on the internet. But you are only asking a question because it is a really hard question.
When you describe the context and the things you've tried so far you will signal to people that you're operating at a level beyond their ability to help.
Asking a high quality question is a universally good move for both newbies and experts alike. It takes more effort, but when you are asking strangers who owe you nothing to put in effort on your behalf it can be a demonstration of good faith to put in obvious effort on your part.
The problem is the guy who neither understands X or Y but just has to answer your question.
No mister, I don't care about Z. And I don't have the time to explain the problem in simple words to you. Either give me answer on X or stay out of this thread.
On SO I have to resist the urge to comment on those types of answers. A good wrong-answer makes the original-post & correct-answer couple better.
Interestingly, that’s not the case with single-threaded forum ‘discussions’, where the y-answer discussion torches your post.
The Q&A-type really shines in this respect, vs the discussion-forum.
Just have to say, XY happens on Arch Linux forum a lot. High level of knowledge of many users, versus lots of peeps interested to try Arch who know enough to install and generally maintain a Linux system.
Yeah, I would expect the XY problem to be more common with Arch more than anything else. You can go from trying to make sound work to scratching your head at GCC errors very fast in Arch world.
Yeah. I hate situations where you have thought about something more than anybody else but you still have smartasses start questioning you after being exposed to the problem for a few seconds.
I am fine with people asking questions but they first should have built up some credibility.
If a programmer whom I know and respect in general asks me how she can do X, I tend to start with an answer about X.
When a name I've never seen before on a -users mailing list asks me how to do X -- and X is bizarre -- I assume we have the XY problem.
We have the XY problem because the Internet is full of newbies at any given point, and we've wasted enough attention-span having sixteen-part conversations about how to extract the IP address from ifconfig output* that we'd really like to know if we can skip all that and answer with "Use a dynamic DNS client, don't write your own."
*Don't. Use ip a
...
Use awk.
...
There are an infinite number of guides to awk, but mostly you need to know how it splits strings into fields and prints the ones you specify. How are you planning on handling machines with multiple IPs, by the way?
I just want to point out that the example is really poor behavior on the part of the teacher: Shouting in all-caps to ASK FOR WHAT YOU REALLY WANT is more likely to result in the newbie not asking at all next time.
If this is your work environment you need to make it okay to ask questions, and to dialogue about a problem without shaming the learner. (Really that should be civil behavior in any environment.)
While it's an important phenomenon to be aware of, its name fails to carry any semantic meaning. Should be named something like "falsely known solution" or "context stripping" or something
Lazy social-signaling drives the over-application of the "XY Problem".
In a private medium, Questioner and Answerer can delve into the wider context and arrive at a well-informed solution.
In public however, any line-of-inquiry is corrupted by its presentation. In the specific context of public Q&A, this means that the motivation of the average answerer is often just to _present_ themselves as knowledgeable without actually being helpful.
In most difficult questions I've seen posted on SO, most answerers have neither an understanding of the context nor the basis with which to compose the "right" answer. Compounding this ignorance is that the Answerer fears losing face and also that he or she is not not personally or deeply invested in the question itself. Thus, a clueless Answerer who accidentally or foolishly interjects in the thread, will eventually default to a simple strategy: negotiating the question.
Less "digging" and more "dumbing", the question is simplified or distorted beyond its original intent so the Answerer can "answer" or rather discount it. They thereby "resolve" their stake in the question posed:
giving them (or at least not hurting) the esteem they crave in the larger group. At the very least, they'll have said something and appeared, in their minds, to be just as smart or smarter than the questioner.
StackOverflow's basic design weights against this kind of child's play, since only the Questioner can award the Answerer the checkmark. But I think the more popular Questions still suffer from the transients desperately seeking their ego-fix.
It makes me think that whole software projects (and popular ones at that) have fallen prey to the XY problem. For instance, are containers really necessary, or are they shoring up a lack of robust dependency management allowing different software programs to use different versions of the same application simultaneously.
Yes, the most common type of XY problems I see is people trying to solve made up problems. It happens to everyone. Lately I was browsing the Google X website [1].
Here's the kind of stuff you find there:
WE CREATE RADICAL NEW TECHNOLOGIES TO SOLVE SOME OF THE WORLD’S HARDEST PROBLEMS
(Why would you solve the hardest problems as opposed to the most valuable ones? Oh well, let's keep going)
How can balloons deliver the Internet to rural and unconnected places?
(I don't know, but are we sure using balloons is really the most sensible solution to that problem?)
How can kites be used to generate electricity in unexpected places?
(I don't know why we'd want to bring electricity in "unexpected places" as opposed to places where it's required, or why kites would be the best solution for that either)
Etc. It's very easy to create very hard problems if you impose unnecessary constraints.
I remember in the early days of phone apps, you'd find thousands of different apps that try to solve a trivial problem. It inspired me to make this video: The alarm clock problem https://youtu.be/ebQAM5ADfYQ
Can someone explain the rationale of setting up a domain for just one blog post?
As well as seeming like quite a lot of effort, the fact domains expire means that content probably won't past 2 years unless you fancy keeping paying for the handful of visitors it'll get.
My rationale for doing this with linuxatemyram.com was so that people would be able to offer this FAQ to someone on IRC or irl without having to Google up that one blog post they once read on that topic.
It also helped focus the message by removing all the notion that this is one of many articles on a general tech blog.
Basically lifted straight from https://mywiki.wooledge.org/XyProblem. GreyCat's wiki is the best Bash resource in existence, but the first example is pretty typical for the IRC channel. There's really no reason to berate people. For example, lots of people have never or very rarely been exposed to anything other than three letter extensions (including most DOS/Windows users), and it's entirely possible that they were trying to do something which only involved three letter extensions. Not every script needs to be a completely generic solution.
I think the article focuses on the real disconnect in it's explanation of the problem. The disconnect being that the step "Others try to help user with Y, but are confused because Y seems like a strange problem to want to solve." is just one extreme in the scenario with the other being "Others ask what it is you're trying to solve instead of talking about the question".
For most questions of the nature each extreme is normally a horrible way to approach it. Something like "You can do it via <method> but there might be a cleaner way to do things depending on what you are trying to accomplish in the end" such as the examples in the page goes miles ahead in being helpful and reduces the overall amount of back-and-forth. This doesn't get a fancy label to reference and articles written about it though as that's just known as a "conversation".
The problem comes in many guises. A similar problem arises when someone says "Y" doesn't work. After going on a long trek you find out "Y" is just fine - they had some other problem "X", had done some diagnosis and decided it was caused by "Y" being broken.
After many years of being dragged down this rabbit hole my first question is now "what makes you think Y isn't working". If the answer is "because X is doing ...", you've just dogged another hours wasted time in another rabbit hole.
This gets reposted now, a day (ish) after I saw someone comment "is this an instance of the XY problem?"
I suspect that people are mining data from ("paying attention to") certain posted keywords and then re-posting.
How might I disprove the null hypothesis that people are just repackaging highly voted comments? How might I convince myself that I am wrong or right? Please show your work or cite sources.
The conflict between describing X vs Y is also about a few things:
- the size of signal: asking a pointed (Y) question saves typing/reading/etc. in some situations
- saving the person time and showing that you came to them after some thought (rather than with the original problem)
Asking Y can quickly become about X even if you begin by asking Y.
This presumes that the asker is intentionally concealing information. I think it's better to assume that they just don't know that some details of their problem might be extremely relevant...
There's a money side to this that's also important.
Had a friend call me a few months back with a technical problem. "How do I fix Y on my server?"
Now I knew that server wasn't bringing in a lot of money, and the Y he wanted to fix could be accomplished the same way with say, 100 bucks of purchasing something else. But I figured it was better if he figured this out himself.
So I asked him what X he was using it for.
That was it. We never got past that, mainly because he was dead-set on spending a week or two of effort and tens of thousands of dollars on something that was (pretty much) trivial.
So after circling the barn a few times on the phones, I gave up. Told him that yes, based on what he was saying he had a fix for Y that was fine. It was just going to take a long time and cost a lot of money.
"But there has to be a quicker, cheaper way!"
Argghhhh.
Yes, there is. But if you can't figure out that you're solving the business problem first, the technical problem second? There might not be one. Technology exists for a reason, and technical problems exist because business needs are not being met. Fix the business problem. It'll drive the tech decisions.
It was quite a frustrating call. For both of us. The X of the XY problem might be critical. Or not. People who ask dumb questions get dumb answers.
I was. We were nice to each other. He's a dear friend. I don't think either one of us were being unkind to the other. I stayed on the call as long as I could. I wanted to help him out. He was just trying to scope down the conversation at the wrong time. I understand his motivations and reasons, but it was premature.
I worked with a really smart leader of a large agency many years ago. Not only a nice guy, he was also brilliant and on top of everything.
I could see several projects he had were going off-the-rails. Why?. Because he kept asking technical questions and getting technical answers. The questions were fine but context-free. The answers were good but context-free.
There was no X. He kept asking Y questions and getting really smart, detailed answers. It was a terrible mess. If you have no context for a technical answer, the only thing an answer is going to do is confuse you.
It's like the old joke about the guys getting lost in a hot air balloon. As they passed over a building, they saw some people on the ground.
"Where are we?"
"In a hot air balloon!"
Now that's a joke, but it hides a truth: without context and a shared language you can get answers all day long to questions and not be any better off at the end than when you started. Probably worse off.
Even though Y was a concrete problem that they do have domain experience in solving!
I call this “going up the rabbit hole.”
——
Example (deliberately shortened from its real depth):
• I’m trying to invent a novel algorithm for inter-procedural data-flow analysis, because none of the existing ones work for me.
• Why? Because I’m trying to reverse a stack machine bytecode with only indirect jumps and no reified CALL/RET opcodes, into a structured inter-procedural control-flow-graph.
• Why? Because I want to do program-slicing to recover HLL representations of key-value-store writes that the program does.
• Why? Because I want to use them as hints for my runtime execution tracer for said bytecode, so it can emit better traces.
• Why? Because I want to put those traces into an OLAP database and I want the data in the DB to be typed.
• Why? Because these traces are the only canonical representation of the state of the system, and our business model is allowing auditors to figure out what’s going on with the system.