MU* Activity Survey 2018 - DRAFT
-
On MSB, we talk a lot about what makes for a more active game. Everyone has opinions and discussions tend to go in circles with the same people making the same points with the evidence provided often anecdotal.
Y'all, I'd like to add some numbers into the mix.
So, I've designed a survey on Google. In creating this survey, I've tried to create something that anyone can answer for any game, whether that game is currently around or was active in the past. It's designed to let both staff, players, and guests respond, because all will have different perceptions of a game, with the truth being somewhere in the middle. I've also tried to balance comprehensiveness with timeliness. In general, the survey should take about 10 minutes to complete.
My plan is to make the data from the survey (stripped of IPs, timestamps, and other potentially identifying information) available to anyone in the community to interpret and use at will. I'd also like to use it as a jumping off point for future surveys.
I'd like to begin taking responses later in the week. Before then, I'd like to make sure I've covered all my bases. I'd like a couple volunteers to read through the questions and let me know if they see any glaring absences.
I will let you know up front, that I have not included a question asking how staff deals with problem players, because I can't think of adequate wording. If you can come up with decent wording for that question, share below! I'll take it.
**Link to the survey is: https://goo.gl/forms/h3Cxeb3AnMTHK0Ij2
-
@lisse24 The one thing that stands out is, if you want to crunch numbers automatically, you might get people referring to games with different variations of their name.
I.e. "The Eight Sea", "T8S", "Eight Sea".
If you do them by hand then yeah, I guess it won't matter.
-
Is the idea that people would take the survey multiple times, once for each game they want to "report" on?
Also, in the game type, some pretty common ones are missing, including Star Wars, Star Trek, BSG, and Game of Thrones in particular (maybe just 'established Fantasy' and 'established Sci-Fi').
-
@seraphim73 said in MU* Activity Survey 2018 - DRAFT:
Is the idea that people would take the survey multiple times, once for each game they want to "report" on?
Yes, and multiple people could provide info for the same game.
Also, in the game type, some pretty common ones are missing, including Star Wars, Star Trek, BSG, and Game of Thrones in particular (maybe just 'established Fantasy' and 'established Sci-Fi').
Thanks! This is an area where I knew I was weak!
-
@lisse24 Minor thing, but there is also some inconsistency on whether you use "the game" or "your game" in questions.
-
A few thoughts:
- One question seems to assume that 'in public places on the grid' means available for RP, but keep in mind when crunching numbers that there are games for which this is not the culture, and where people are pretty much never out in public unless RPing.
- Your 'low XP or high XP' question ignores games with no XP or no system at all.
- Your 'PvE or PvP' question ignores games with neither (I might add 'social' as an option).
- How is a non-WoD game meant to answer the single sphere vs multi-sphere question? Are you really asking about factions?
- What is the different between a political game and a plot driven game? What if the plots are political?
- Really every question should have a 'NA' or 'other' option. I've run several games with no '+request' system, and one with a system that no one ever used really. Your questions about how information is disseminated are also likely to turn up some other weird options.
- Consider adding additional terminology for 'temp rooms', especially with the advent of Faraday's scene system.
-
You also ask about +requests but never ask about applications. The length of time it takes to apply and how hard/easy it was perceived to be does matter for activity on the game.
ETA: Nevermind the whole email application or email required for application debate.
-
What I'd like to know is, removing identifying information as indicated, how often staff and player perspective differentiates for MU*s submitted (assuming staff and players submit for the same places).
-
"Approximate average percentage of players engaged in RP at any point."
I don't know how to determine this. I could code a system to guess, but from a perception standpoint, how does anyone decide this?
“If you wish to make an apple pie from scratch, you must first invent the universe.”
― Carl Sagan, Cosmos -
@thenomain Ultimately a system is going to have to do some guessing unless it is overly obtrusive, but you can probably make a pretty good guess.
What I would do is code a system that checks every 5 minutes or so and figures a person is 'engaged in RP' if they are in an IC room, have been idle for less than 15 minutes, and there's at least 1 other character in the room who has also been idle for less than 15 minutes. If all those conditions are met it considers the person to have engaged in 5 minutes of RP (and since the check occurs to everyone it would consider the other character to have also engaged in 5 minutes of RP).
Since everyone's mileage varies you might want to turn up or down the length of time someone's been idle (or even make the length based on the number of people in the room since people will usually idle a bit longer in big scenes as they wait their turn in the pose order) before it is counted as 'RP'. If you are have connection to an external database such as one on a mySQL server I might not even do that. I might just record each entry at 5 minutes as character, location, and idle duration. Then I can tailor and adjust my queries on the raw data until I think I'm seeing pretty accurate results.
I've written a couple of these types of loggers before and found them to actually be pretty useful. With a little cross-referencing of data you can see what the active days of the week are, hours of the day are, what locations show lots of activity, and how active one sphere is relative to another.
-
A large part of my point is that this makes it a question for the people running the game, not the people playing it. I'm sorry to say that the question also centers around my core question: What is the purpose of this poll?
“If you wish to make an apple pie from scratch, you must first invent the universe.”
― Carl Sagan, Cosmos -
@thenomain said in MU* Activity Survey 2018 - DRAFT:
What is the purpose of this poll?
Yeah, I don't mean to be overly negative, but I think studies/surveys like this work best when they have a thesis in mind. Otherwise it's just a random collection of disjointed, inconsistent and intensely subjective data that IMHO will be impossible to draw any meaningful conclusions from.
-
@faraday (Raw) Data itself is not really subjective. The biggest danger you have is that it simply won't have something you later realize you needed to capture. This is why I'm a fan of the broader capture form of character, location, and idle duration (and I should note that I'm also making some assumptions such as that you will be able to reverse engineer certain other points such as the player the character belongs to and the sphere the character is in. If that's not going to be the case I would probably capture those bits as well).
The truth of the matter is that if you are storing the data in any kind of real database (i.e. not inside a game object or something) the data storage is pretty cheap. For a game that polls data every 6 minutes I should be able to take Timestamp, Character DBREF#, Location DBREF#, Idle, and Sphere of each person online and it will take about 1 MB per person (so if you have 50 people connected on average about 50 MB) a year to store that data. You'll pick up a little more memory if you do things like create a table so you can connect names to the Character and Location DBREF#s, but not much at all.
Now the interpretation of that data certainly can be subjective. Are your parameters really showing you something reasonably approximating RP? That gets a lot harder to say. Even worse, it could be argued that even if you get the magic formula that shows you exactly how much RP is occuring that may not ultimately be the best way to determine the 'health' of the game. Do people just hanging about the OOC rooms chatting or talking on channels count for anything?
-
@the-sands said in MU* Activity Survey 2018 - DRAFT:
(Raw) Data itself is not really subjective
I think we're talking apples and oranges here. Activity metrics captured from an actual game in an automated fashion are, of course, objective. The questions in this survey (which is what I was talking about) are subjective.
-
@the-sands said in MU* Activity Survey 2018 - DRAFT:
@faraday (Raw) Data itself is not really subjective.
I don't think this is true, especially in regard to polling. How you frame questions and what questions you include, has a huge impact on the data you get. We also aren't really talking raw data, here, we're talking a survey with voluntary participation, which in itself already does things to your data set that you need to take into account. I DO think stuff like this is useful, mind you. But it's not raw data on MU community activity. Stuff like the mudstats.com activity graphs are, and I still love looking at those tables because I enjoy data nerding and think it's useful, but that site doesn't seem to add or delete games with any regularity, alas.
-
@tat said in MU* Activity Survey 2018 - DRAFT:
A few thoughts:
- One question seems to assume that 'in public places on the grid' means available for RP, but keep in mind when crunching numbers that there are games for which this is not the culture, and where people are pretty much never out in public unless RPing.
- Your 'low XP or high XP' question ignores games with no XP or no system at all.
- Your 'PvE or PvP' question ignores games with neither (I might add 'social' as an option).
- How is a non-WoD game meant to answer the single sphere vs multi-sphere question? Are you really asking about factions?
- What is the different between a political game and a plot driven game? What if the plots are political?
- Really every question should have a 'NA' or 'other' option. I've run several games with no '+request' system, and one with a system that no one ever used really. Your questions about how information is disseminated are also likely to turn up some other weird options.
- Consider adding additional terminology for 'temp rooms', especially with the advent of Faraday's scene system.
I'll certainly use this feedback, but my fear with adding a N/A option is that, say, you'll get someone from a game that uses XP, and they'll scoff at both options and choose N/A. When you're trying to identify trends, you really want to force a choice, even if that choice isn't perfect. However, I think that I can probably word a third choice to avoid that. I'll work on it.
@lotherio said in MU* Activity Survey 2018 - DRAFT:
What I'd like to know is, removing identifying information as indicated, how often staff and player perspective differentiates for MU*s submitted (assuming staff and players submit for the same places).
I'm also really interested in seeing how that differs.
@thenomain said in MU* Activity Survey 2018 - DRAFT:
"Approximate average percentage of players engaged in RP at any point."
I don't know how to determine this. I could code a system to guess, but from a perception standpoint, how does anyone decide this?
Right. This is absolutely 100% perception. What does the person think is going on. There's no way to actually measure it, but I don't think it's necessary. I'm willing to bet that players and staff's perception of activity is pretty darn close to accurate.
-
@lisse24 said in MU* Activity Survey 2018 - DRAFT:
I'll certainly use this feedback, but my fear with adding a N/A option is that, say, you'll get someone from a game that uses XP, and they'll scoff at both options and choose N/A.
The only real way you can prevent this is by pre-populating the options for MUSHes in advance. So we'd be picking from a list of them (instead of typing them in), and this information would be consistent.
Obviously it involves a lot more work on your end.
-
@arkandel I was thinking of adding an option like: Game does not use XP, Game does not have a grid, etc. So people on games that don't use that system can opt out, but still force the choice for people from games that do use the system.
-
@lisse24 Something else you can do in general is add an "I'm not sure/I don't know" option. It beats making it up if it's an otherwise mandatory answer.
-
@lisse24 said in MU* Activity Survey 2018 - DRAFT:
I'll certainly use this feedback, but my fear with adding a N/A option is that, say, you'll get someone from a game that uses XP, and they'll scoff at both options and choose N/A. When you're trying to identify trends, you really want to force a choice, even if that choice isn't perfect. However, I think that I can probably word a third choice to avoid that. I'll work on it.
There are a lot of your questions that flat out do not apply to most of the games I've run or helped run - which is 4 or 5 depending on how you want to count. So while I get your fear, you should ALSO have the fear of 'how accurate is my data if people are choosing things that straight up don't apply to their game because they don't have a correct option?'
I highlighted most, but not all, of the questions that seemed to have that particular issue, but I'm just speaking from my viewpoint, and I imagine that other games exist for which different questions might also feel wildly inaccurate.
For what it's worth, I think an 'NA' option is probably less likely to be abused than an 'other' option, but also I think most people are lazy and will choose a ticky box over filling out 'other' where it seems feasible to do so.
