Studies

some gaita sanabresa. today this post is shaped like a scientific article well, because it is. if you want to see the results first check the link in the abstract. apparently, CS is not that misogynous after all.

abstract

much has been said about gender bias in hospitality networks, and much of it has been mostly speculation. it has been frequently said that males on these networks exploit them as a way to meet females. therefore, i hereby present a quantitative analysis of a specific case study: the rate of replies to males and females on last minute groups on the website CouchSurfing. the final charts can be seen on this page

methodology

all data collected was done so via publicly available data, much like a search engine would do it. in the cases where users choose to make their data private, or groups choose to require a login, no data is collected.

the data was collected using a data mining technique, using PHP as the main language. this data was then dumped onto a TSV file that contained the timestamp of collection, number of replies, the gender of the person sending the message, and the country and city of the group. group id, message id and user id were also collected for consistency.

data set choice

last minute groups are characterized by being focused on a city or country, so local groups were chosen from various global locations. the software was developed so that more groups can be added easily. in this case, 36 groups were chosen. after mining, 22 groups were left from 18 different countries. data from blocked groups was not gathered since it is not publicly available. 5 pages from each group were collected, totaling 1760 messages.

data analysis

once the TSV file was populated, it was then fed to a SQLite relational database, with a primary key on the postid, so that duplicate posts were not taken into account.

the data was then grouped by gender, gender-country, and gender-country-city tuples, calculating the rates of reply, maximums and averages. the data was then processed into an HTML page that charts all of this information. note that when gender is unknown that can both mean the user did not fill it out or has chosen not to have it publicly available.

for visualization, a bar chart was drawn with the maximum length as the maximum average response rate of all groups by category, allowing for visual comparison of different groups or countries directly by visual inspection.

results

the results can be seen in this html page or embedded below if your browser allows it

discussion

world data

the world data was very leveled, indicating that there is little gender bias in general, though women do tend to have a slightly higher response rate than a general user. this indicates, though not strongly, that being of the female gender favors the response rates expected by a user on this network.

country and city data

the country and city data is perhaps the most interesting part of this study, as very strong contrasts are seen between different groups. for example, India showed 3 times the average response rate to females, versus 1.4 times the average response rate for males, indicating twice the response rate to females versus males. on the other hand, Luxembourg had 3.33 times more replies to males than females (which were 1.81 times the world average), signaling that in Luxembourg it was twice more likely for a male to get a reply than a female. in general, cultural differences are very strong between countries in regards to gender bias: some countries are very gender-sensitive, while others are not significantly sensitive.

conclusions

this demonstrates what is frequently said about gender bias in a global perspective, that females are favored by their gender in finding a place to stay. however, if the data is split regionally, this no longer adds up. the culturally different attitudes towards gender seem to be stronger than what common sense would claim, disproving the hypothesis that females are always favored by their gender. females are favored by their gender in some cases, in others not, so this is sufficient to disprove that there is widespread gender bias on CS. it exists, but it is confined to specific geographical locations, with wildly varying amounts, and not in a very significant way at all.

note that this data is biased by the fact that we only analyze people that couldn’t find a couch and are using last minute groups, which in itself cannot be used to generalize further, though it is already indicative of no significant global gender bias.

sources for replication

i provide all sources and the database collected for download freely, as long as the license of this website is respected. all scripts are prefixed with a shebang line so that they can be used in a shell environment. if your php-cli is located elsewhere, you should change that line. code is not commented, it is too small to be complicated, but if anyone needs assistance just comment below.

• [SQLite relational database file](http://ubuntuone.com/p/raR/)
• [TSV file of the data mined](http://ubuntuone.com/p/raS/) (works with excel/open office)
• [data mining PHP script](http://ubuntuone.com/p/raU/) (includes a simple way to add more groups. it randomizes the request interval to avoid firewalls and cause server problems)
• [TSV to database script](http://ubuntuone.com/p/raa/) (converts data from one to the other, effectively filling up the database)
• [HTML code generator](http://ubuntuone.com/p/rab/) (generates the html graphs from the data present in the database)
• [HTML output example](http://ubuntuone.com/p/rac/) (the output of the software for the data in this study)

comment

it is a bit unsettling to me that all this data is made publicly available by CS without any control. this has to do with the default privacy setting: people share everything with the world unless they choose not to. this means that most people that aren’t particularly tech savvy will end up sharing more than they would expect. this is becoming a trend in online websites, share everything by default, which, in my opinion, sets a dangerous precedent. not everyone is like me and is doing this to test out scientific hypotheses. this information can be easily exploited commercially, with some 30 mins of coding like i did.

this study also demonstrates how informatics can be helpful in social sciences, and that with a little bit of coding one can get huge datasets automatically, ready for processing. it took me 30 minutes of coding to set up the mining, left it overnight to crawl the website, and then about 1h to setup the visualization. it should be interesting to see this data on a map, but for the sake of my free time, i’m not going to do it.

asturian and galician gaita. CS means couchsurfing

recently CS changed their search algorithm. almost instantly i went from a couple of requests a day (1 to 3) to zero. in fact, now we (me and T) get less than 5 a week the two of us combined. this we can only explain by the changes in the algorithm. but what this means is that i can’t generate enough data to be statistically significant, therefore, i’ve concluded my studies on requests and stays. what remains is data analysis of the past (which is already a huge dataset).

this is also a reflection on the current state of CS. the quality of its members is decaying at inverse proportion with the number of members. we now live in a much nicer place and we get more people creeped out than we did in the previous two (yes, that includes the dog-shit-everywhere squat). with this change in algorithms, it’s interesting to see what will happen.

previously, there was a positive feedback effect on being a good host or guest: you’d get listed above and with it, you’d get more requests and/or more hosts. this meant that everywhere you’d find nodes of CS where you have few, very passionate and active members, with an enormous quantity of references and experience.

with the current algorithm, as far as i could investigate, they leveled the field for everyone. i agree with the idea behind it: allowing everyone to be able to host and surf as easily as everyone else. but my guess is that this will bring the average stay quality down, just by exposing guests to everyone, rather than “professional” hosts like top hosts usually are. they replaced a meritocracy with a democracy.

i expect to be slowly (and naturally) marginalized as time moves forward with this algorithm, since lisbon now has over 2000 people registered, making me a 1/2000 voter in a 2000 population, irregardless of the fact that i’m among the top 50 hosters in the world right now (in 2000000+ people, i was the 14th most experienced, considering data from today). my contribution to this community will slowly be eroded as time goes by. i have ambiguous feelings towards this that haven’t matured yet so i don’t really know how i feel about this. i guess it’s good to kick out the bittered hosts, since all hosts bitter up at some point after too many guests (any good data on this?).

i also noticed a bias towards people that choose to make their personal information public. people that show off everything about themselves to the world (including google), will get listed over other people. this is an interesting tweak that probably offsets some of the effect i described above. CS has always capitalized on ego and self promotion, so maybe this shift will upset some users, but actually make the website more usable. we’ll see. but for now, no more experiments on request rate and so on.

i’m moving closer and closer to a normalized database, and with it, many new stats. one of the ones i wanted to see was the distribution of guests by country. i noticed early on that i seemed to get guests mostly from rich countries and no guests from africa for example. so i also cross referenced it with the GDP of each country. there is a relation, no doubt.

these are the results for the total amount of guests. i didn’t do stats on uncertain origins.

zoomed in:

these are a bit more interesting, how they relate to GDP (sorry, had to put the legend in the middle so it wouldn’t cover points, and it’s so big it didn’t fit).

in both cases we can see that there is a great “divide” between the high GDP – high traveling countries and a lot of poor visitors on the bottom.

you can get the R source code and the source data and replicate my results. remember the data is licensed (see license on the bottom of the page). i will progressively provide more stats while i normalize the database.

what i see here is just another obvious fact. rich people travel more, poor people just can’t do it. couchsurfing might enable people to travel using less money, but it’s failing at getting poor people to join it. think about it, who has access to the internet and enough money to travel? i don’t want to be simplistic, but i do think this is food for thought. in a way, it is a hospitality network not for those that need it, but for those that don’t. just think about that taboo on couchsurfing: never say you’re short on cash! i guess you can’t be poor and acknowledge it.

this is apropos, we had our first money theft in a house (at _42). i guess it was only a matter of time until it would happen.

6 out of 26 guests have left _42 creeped out (23%, or a bit over one out of five). we may have underestimated the importance of a couch description. our couch is significantly better than the two previous ones. we also chose a more artistic description, because we are working actively on design and engineering projects all the time.

the problem, it seems, is the same old problem about people. people describe themselves as what they want to be, not as what they are. close minded people will invariably say they are open minded. racist people will say they “love multiculturalism”. intolerant people will rave about how tolerant they are. to quote bukowski,

and the best at murder are those who preach against it

and the best at hate are those who preach love

and the best at war finally are those who preach peace

those who preach god, need god

those who preach peace do not have peace

those who preach peace do not have love

full quote here, ironically the second post on the e8 blog.

the real honesty comes from acknowledging our own hypocrisy and fragility. to know that we are all fuckups in a way or another. this is an instinctive natural behavior very common in travelers, since they are very dependent on others. but that is disappearing from cs too.

the problem here is that couchsurfing is slowly changing. the average social class of our guests has been going up (again, maybe due to a more “clean” description). with it, the disgust faces, the awkward moments, and the people leaving when we are not there.

so we changed the description again, to something scary, like it was with SPCC and e8. this means we will be biasing our data heavily, but that also means we won’t have to put up with arrogance and narrow-mindedness veiled as fragility or generosity.

we are a bit tired of guests saying they are leaving because they are sick when they are disgusted. that they are leaving because they found a friend when they just don’t want to make friends. that they are leaving to make room for others, when what they need is more space for themselves.

once we get to minds (i know, i’ve been saying this a lot), it should become clear how this happens so much. for now, let’s just hope being scary brings us less awkwardness.

we’ve resumed our data-centric analysis at our new mandala (_42, no website yet). one of the works being done now is getting the whole data set ready for statistical analysis. almost done. but the funniest thing was the graph of how i liked other people, subjectively. in 570 data points, guess what, i got a bell curve. this means, so far, the best way to model encounters is a random variable. we are slowly separating variables from each other, so expect a lot of charts soon.

so for now, here is the raw “hate to love” chart, 1 being hate, 5 being love. fits my prediction almost exactly, to all my guests that doubted my graphs on the wall tiles. is this confirmation bias? i don’t think so, because there seems to be a negative bias on my part. this fits psychology, that humans have a negative bias. we are also going to add further ratings by different people, to clean up the personal effect. this will only work for about 100 data points, but should be good.

data set: 570 guests

one of the key studies will be sexism on cs. we’ll add more info about this soon. we will change genders every week and see how that affects requests (gender bias and quantity). we are also working out how different spaces affected scores and other things. we are on our third place. one of the hypothesis i want to test is whether the place itself changes the people we meet and how we get along.