Scheveningen Posted February 25, 2020 Posted February 25, 2020 (edited) Several community polls have been made in the past where they present several options/adjustments to an issue with only marginal difference between the vote options, or there are more options than needed. This does not make for very good community feedback (in the sense that it cannot be relied upon as easily) when the overall votes are spread out across several different vote categories. In most cases this makes certain voted decisions look more or less invalid than the others depending on who voted for them, due to the bloat of decisions you can make in a single poll. This shouldn't be the standard. A simple "yes" or "no" with a given subject should do. No "yes, but" option or its cousin "no, but" either, because that's stupid, and most questions pertaining to polls should be framed in such a way that they're not open-ended past "yes we should have this" or "no we shouldn't have it." Common ground based on vote results end up being figured out by the trusty staff anyway, but binary decision-making with polls makes it feel like it actually makes a difference to contribute in said community poll. There should not be a neutral vote either, as personal indifference to an issue should be marked by not voting in the poll. If you don't vote in any meaningful way, you shouldn't be counted. Edited February 25, 2020 by Scheveningen
Skull132 Posted February 25, 2020 Posted February 25, 2020 This will not be codified. Because it is a dumb thing to codify. What you plan on getting feedback for will dictate how you set up the polls and what questions you ask. It is up to the decision maker to figure out what question he wants to ask, and how he wants to interpret the answers. Specially considering that all community polls are non-binding in nature. Sometimes a very clear binary choice is the most reasonable option. Sometimes degrees are permitted, with, if necessary, some inbetween choices being categorized as a super-group for a more cut-and-dry view on the matter, if it proves necessary. Obviously yes, there are certain practices in polling and statistics that might be beneficial to adhere to, but I assure you of two things. What you wrote is not clearly present in them, and the list of best practices is too long to actually codify.
Scheveningen Posted February 25, 2020 Author Posted February 25, 2020 (edited) Quote "The quality of a survey is best judged not by its size, scope, or prominence, but by how much attention is given to [preventing, measuring and] dealing with the many important problems that can arise." --"What is a Survey?", American Statistical Association Ensuring quality and propriety standards exist in surveying/polling and allowing those standards to persevere through time is never a dumb thing to codify. We don't need the Ten Commandments to agree that there should be good reasons for doing anything. 1 hour ago, Skull132 said: What you plan on getting feedback for will dictate how you set up the polls and what questions you ask. Assume for a moment that polls are of course created by individual humans with their own individual wants, that make them ever so slightly different from other humans with their own wants. The matter of surveys face the constant issue of matching question wording with concepts being measured and the population being studied. The ideal survey recognizes that the most vital stage of posing surveys is the initial format of planning the question(naire) - but it is not the question only that is important, it is the answers you get. It stands to reason that the issue is not the matter of formatting questions to be appropriate and precise in what they're asking, but rather that formulating potential answers/solutions should not throw off a community from being able to precisely answer a survey. Such as with adding multiple-choice answers inappropriately to certain questions polling for opinions. In questionnaires formatted for academic testing purposes, multiple-choice questions are designed to make it more likely to compel a test-taker to fill in the incorrect answer should they not know the correct way to solve a problem, whether mathematical or otherwise. The query fills you in with several factual details and implications as to what the formula might be and what the methodology involved might also be. Because, simply, if you don't know how to solve a formulaic problem, it should be left roughly to chance that the test format should penalize you for not having read the material necessary to pass the test. In essence, with questionnaires like that, there is one right answer and every other answer is wrong, and those who don't know how to problem-solve and get the right answer should have to deal with the stacked odds of getting a wrong answer. This is ultimately quite a different thing from polling for opinion, however, where multiple choices of answers can serve to be harmful when polling for population preference. As mentioned, it is the inverse situation with polling matters of opinion, particularly to gather what a population might prefer. In regards to surveying preference of a population, the right or wrong answer simply does not matter, it is not the objective of the question - the goal is ultimately to find what the overall population prefers or doesn't prefer. Whatever decisions are made with that information is as a basis doesn't really matter since I imagine there's trust in people to do what they're asked to and have results come of that. Now, I actually wasn't entirely accurate with saying the right or wrong answer doesn't matter. There exist wrong answers, and those are "unrealistic/unfeasible scenarios". Such as waxing hopefuls what could be but would never actually happen due to a variety of reasons, or just one. Those are "wrong" answers in my mind, and have no place in the question or answer part of a poll. Polling forms a basis of "doing things" versus "not doing things", empirically truthful as what is the reason for doing something about something if no one is to suggest improvement of a thing, or when there is no complaining about said thing? Fixing what is not broken (or that which does not need improvement) often leads to wasted time and energy expenditure. A simple complaint lodged through a staff member's private messaging, or that of an official complaint, is of grain-of-salt sample size and reliability feedback - yet can still be considered 'polled' of a single sample size of one person. An additional vote of confidence - or lackthereof, which is not entirely useless even if it just 'one more' or 'one less'. Combine this with other prior information based on the subject of what is being reported. Dependent on the process that comes after having received said feedback, action is either taken in some way or not at all, dependent on how reliable the information from said feedback was. Indirectly, these are all methods of polling and when collectivized become very useful bits and pieces of information together. Time to digress. Speaking of things that are broken. I do not believe multiple-choice to necessarily be the devil, but posing questions with their related answer possibilities with the format of, "Yes, do the thing", "No, don't do the thing", "Do this entirely unfeasible/unrealistic thing", or "How about this other entirely unfeasible/unrealistic thing" - is pretty much 'doing evil as if.' In essence, the population should not be fed the possibility of answering in such categorical ways that aren't something the decision-maker(s) is/are at all willing to do - or, something that simply isn't physically possible, such as hosting the Aurora server on the moon. Likewise, introducing too many necessary variables (i.e., unfeasible answers to a poll) makes a situation more complex than it needs to be. Remember the 'stacked odds' thing? It plays a role here. Another example, the "rework, not remove" issue. Despite that the overall population of the community tends to vote for reworking things instead of removing them (due to the stigma involved of removing things, even if said things are bad as they stand), what if the decision-maker wants nothing to do with 'reworking' something they dislike and would rather spend their time on more perceivably useful endeavors? Therefore, why would the decision-maker even put in "do a thing I would never personally be motivated to do" as an option to poll the community for anyway? It makes no sense. Thus, my suggestion to make polls as binary as possible - but I am not discounting the possibility of third or perhaps fourth choices. Not every possible scenario one can think up is either realistic, or is anything that anyone wants to go through with on their part. I genuinely believe polls can be made more meaningful if the polling choices are condensed to minimum of 2 technically and equally feasible choices - because that is obvious. I am not against the idea of having three or perhaps even four different routes to diverge from on a single particular issue, but all of the possible decisions should be as equally valid and feasible as the others to execute. And that is the point of the policy suggestion, to ask that we gravitate far away from unrealistic theoretical outcomes being posed in community polls as options to vote for. Non-answers should simply not exist in said polls. I can think of a fair number of people who wouldn't fall for 'trap answers' that are so clearly unfeasible and unrealistic, such as 'rework, don't remove', but it still doesn't stop the possibility that at least someone will anyway. Edited February 25, 2020 by Scheveningen
Skull132 Posted February 26, 2020 Posted February 26, 2020 Well, that's an essay. But I'll cut straight to a point. 6 hours ago, Scheveningen said: And that is the point of the policy suggestion, to ask that we gravitate far away from unrealistic theoretical outcomes being posed in community polls as options to vote for. Non-answers should simply not exist in said polls. The whole deal that echoes through your post, but is highlighted here, is that you seem to think you know what is useful to the survey runners. Basically you are attempting to impose your own view on the matter. Surveys are a tool for gathering information, nothing more. Their efficacy is not tied to how realistically attainable the options listed there are, they can easily be used to poll emotional understanding or wishes and hopes -- theoreticals. Your view completely annihilates this possibility. This is not productive. All of this starts falling apart when I ask, "What if we want to know if the playerbase would be interested in a rework?" For example, the ling and malf surveys, the prevalence of the theoretical rework option demonstrates that the playerbase is at least interested in the core ideas of the modes, and simply disagree with their present execution. Which is a completely valid piece of information to obtain, and one which would not be obtained by a simple, "Yes, remove; no, do not remove" poll. Another piece of information that can be interpreted from it is that the majority of the playerbase (rework + no votes) disagree with the present implementation of the modes. Which, if you wanted to call for removal, would be the one you would use. So, with this form of processing, I would not consider the answers we gave as "wrong". As for this. 7 hours ago, Scheveningen said: Ensuring quality and propriety standards exist in surveying/polling and allowing those standards to persevere through time is never a dumb thing to codify. We don't need the Ten Commandments to agree that there should be good reasons for doing anything. Yes, it is. We are a loose organization that has no real legal responsibilities towards standards, nor the capital required to enforce these standards. What you effectively request is that we write an SOP for running surveys. What you have to ask for every document or regulation added is: "Is this actually going to be used?" The fact is that 75% of all auxiliary documentation, such as this one, would be put on the wiki somewhere, and then forgotten about. It's really just how this all works. So no, this information would not be useful to document nor to codify.
Scheveningen Posted February 26, 2020 Author Posted February 26, 2020 2 hours ago, Skull132 said: The whole deal that echoes through your post, but is highlighted here, is that you seem to think you know what is useful to the survey runners. Basically you are attempting to impose your own view on the matter. Ad hominem. You assume to know what my intent is behind this policy suggestion beyond that which was already posted. Recall for a moment you speak to human beings on a daily basis who probably appreciate it when they are not jerked around or aren't accused of conspiracy-theory tier woo. Maybe maintain that level of courtesy here, whether you can distinguish humanity over a screen or not. 2 hours ago, Skull132 said: Surveys are a tool for gathering information, nothing more. Their efficacy is not tied to how realistically attainable the options listed there are, they can easily be used to poll emotional understanding or wishes and hopes -- theoreticals. Your view completely annihilates this possibility. This is not productive. Survey answer fields in human research generally avoid potential non-answers and unrealistic hypotheticals wherever possible, so as to not render eventual data completely unusable. There is also quite a bit more to it than gathering information in regards to surveys. Unless you are referring merely to your own perception of what surveys are. In which case, sure. 3 hours ago, Skull132 said: All of this starts falling apart when I ask, "What if we want to know if the playerbase would be interested in a rework?" For example, the ling and malf surveys, the prevalence of the theoretical rework option demonstrates that the playerbase is at least interested in the core ideas of the modes, and simply disagree with their present execution. Interesting, except the two game modes in particular have specific themes as a result of their execution or features-as-present. The ambitious ideal of "can we poll the community if they want a rework or removal of X" needs to take a backseat to pragmatic foresight of "do we have anyone who wants to rework X", first. Polling for whether or not people want to keep and rework a game mode is rather pointless if no developer will step up to do it in the first place, whatever their reasons. 3 hours ago, Skull132 said: What you effectively request is that we write an SOP for running surveys. The only thing you need to add to that list is "Don't add stupid nothingburger answers to surveys." That is in essence all that is being asked above anything else. Do you have to make this so difficult to level with you?
Arrow768 Posted February 26, 2020 Posted February 26, 2020 Voting for dismissal. Restricting polls to binary options is in my opinion too restrictive in a lot of cases.
Recommended Posts