Online content curation, i.e,. things I’ve learned moderating racist, sexist, and homophobic topics.

This is something I plan to post as a moderator for /r/ChangeMyView. However, I take a personal interest in online curation and content moderation. Therefore I’m posting it here as well.

There’s a topic I’ve been thinking about since the lead-up to the 2016 election¹: does CMV create a safe space for abhorrent views? Do we normalize and reinvigorate conversations already rejected by society? What is our responsibility, as a platform, with respect to each of these issues?

A competing response from CMV users is that we create a platform for some really bad views — the kinds of views traditionally considered shameful — even through the lens of diversified perspectives. Some examples are white supremacy, “biotruths” regarding race and gender, and categorizing homosexuality as a mental disorder. I would personally argue these views are rightly considered shameful. My personal views, however, do not grapple with best practices for moderation.

These are conversations that have been explored copiously, from nearly every angle imaginable and, in some cases, ratified by law only to be turned over by democratic processes or formal adjudication. In this case, one wonders, myself included, if there is any value to be had in relitigating these issues. In fact, there is a heightened concern that creating a space for these conversations allows them back into civil discourse, legitimizing and normalizing them. This, in turn, potentially gives them space to breath and ultimately flourish.

Caning of Sumner
Editorial lithograph depicting the caning of Charles Sumner in the US Legislature, 1856. Image taken from

I will flatly admit that I share this concern, especially in 2018. Political and cultural hostility is a tale as old as time (see also Adams and Franklin phhhbbtt). The reason why Internet platforms have come into the forefront is, I believe, for two main reasons: first, the speed at which we are bombarded with information has dramatically increased, and; second, the barrier to publishing information has significantly diminished, thus creating a wider breadth of comments/claims. The latter is extremely hard to moderate logistically, let alone discretionally. Taken together, this creates a unique manifestation of an otherwise old problem.

We have preliminary data showing some previously shunned views are now emboldened. While polarization is not direct evidence for specific views, they are a useful proxy from which we can make fair extrapolations. Other online forums for political debate are deeply partisan. White people and black people have fundamental disagreements on police brutality. Pew Research indicates Muslims fear intimidation – defined as a reasonable expectation of bodily harm – in numbers surpassing the immediate 9/11 era. Increased hostilities have not been limited to the United States.

None of these flash-points point to egregious views per se, but I’m highlighting them to indicate that, to the extent people have views, they are trending more extreme, and that these extreme views are likewise painting a portrait to others, the culminating effect of which is to make them fearful or, at the least, deeply anxious about their relative placement in society.

Pivoting back to reddit, various other subreddits have been deconstructed with an eye towards low quality and/or highly polarized online discussion. Maintstream outlets have explored the intersection between extremist, often “alt-right” political views and reddit:

(Unfortunately, but topical, the FiveThirtyEight article warns for slurs, as will I.)

And that top five isn’t exactly pretty, though it does support the theory that at least a subset of Trump’s supporters are motivated by racism. The presence of r/fatpeoplehate at the top of the list echoes some of President Trump’s own behavior, including his referring to 1996 Miss Universe winner Alicia Machado as “Miss Piggy” and insulting Rosie O’Donnell about her weight. The second-closest result, r/TheRedPill, describes itself in its sidebar as a place for “discussion of sexual strategy in a culture increasingly lacking a positive identity for men”; named after a scene from the “The Matrix,” the group believes that women run the world and men are an oppressed class, and from that belief springs an ideology that has been described as “the heart of modern misogyny.” r/Mr_Trump self-describes as “the #1 Alt-Right, most uncucked subreddit” — referring to a populist white-nationalist movement and an increasingly all-purpose insult meant to denigrate others’ masculinity — and the appallingly named r/coontown is the now-banned but previously central home to unrepentant racism on Reddit. Finally, coming in at No. 5 is r/4chan, a subreddit dedicated to posting screenshots of threads found on 4chan, where many users supported Trump for president and where the /pol/ board in particular has a strongly racist bent.

Emphasis mine.

To CMV’s credit, our media deep dives have been positive², but that’s not self-executing. It takes a lot of introspection about our role, rules, and moderation framework to create the necessary forum – and I do believe it is necessary to have these conversations – where we can talk about all of these tough issues without glorifying the underlying views.

The risk of validating views absolutely exists. I do believe that, as a moderation team, we should be mindful of how our curation impacts the perception of what reddit audiences believe to be fair-minded, critical conversations. I don’t purport to represent the whole of the moderation team in this respect, but I do think it’s safe to say we share an acute awareness of how our impact here grows as our subreddit does.

The place I’ve come to is:  people very rarely want benevolent views changed. CMV could not exist as a place for critical discourse if we tried to create a superficial impression that people only have pedestrian or typical views. On the contrary, we consider it more likely that someone is trolling or looking to push a view (“soapbox”) if they come in with a view most people share. It begs the question why one wants that view changed. I get why a self-identified white supremacist wonders why s/he might be wrong; most people, at least publicly, advocate for equality. That is a clear catalyst for questioning one’s position.

But why would someone want to change their view on, to use a real example, equality of races? Is it possible? Sure. Is it probable? I don’t think so, and while we wouldn’t remove this thread automatically, it would likely deserve higher scrutiny. It is suspicious in the sense that there is an even better probability that this person is posting the opposite of their view in the original post so they can bolster and increase visibility of that opposite view in the comments.

With this heightened likelihood of publicly sharing a “shameful” view, I would rather create a place of deliberate contemplation and critical thinking where this view is exposed to contrary evidence. Our value system ultimately drills down to engendering an ethos of thoughtful critique. We don’t always succeed at this but the moderators do try to be self-critical, regularly step back from our work, and revise our method based on shared learned experiences navigating these waters.

It certainly creates a risk of normalizing these views and creating a “safe” space,  but only if we insulate these users from the civic consequences of their behavior. That is, it is only “safe” if we allow them to share without engagement. Ours is not a safe space for like-minded people in the sense that we advocate the same substantive views. It is a safe space for people to talk candidly and respectfully, to forcefully push-back on ideas so long as one recognizes this is a two-way street of presuming good faith until proven otherwise. It is perhaps our most valued first principle, and it is the compact made between users and moderators each time we participate in CMV.

My view hasn’t changed 180 degrees, but that also captures an element we embed in our delta system: having a good conversation doesn’t require a “gotcha” moment where you lose and your entire worldview is toppled; it simply requires gaining an additional insight that impacts your view. Where I’ve landed is that I am cautiously guarded when I see these threads advocating a “bad” view, but I nevertheless operate by the Principle of Charity: I assume good faith and rationality until a poster gives me a reason not to, and leave it to our users to thoughtfully and creatively critique those views. My responsibility is to curate and cultivate an environment that reinforces an understanding of nuance, but also a willingness to speak up when something is clearly abhorrent and explain why someone should change their mind.

¹ Posting about these thoughts was spurred, in part, by this podcast conversation between Katie Couric and Recode’s Kara Shwisher. It echoes a lamentation I hear frequently about Internet conversations, which is that they lack nuance and fail to appreciate the multi-faceted qualities of just about every human being.

² “Our Best Hope for Civil Discourse Online Is…Reddit” by Virginia Heffernan,; “On the Other Hand” by Tim Adams, The Guardian; Our Minds Have Been Hijacked by Our Phones. Tristan Harris Wants to Rescue Them” by Nicholas Thompson,

An impromptu primer on how to deal with YouTube tosspots.

Listen to Kara Swisher’s interview with Susan Wojcicki (CEO of YouTube). There is a portion regarding rules/codes of conduct/community guidelines. I want to memorialize my thoughts here.

I’m not going to put words in Wojcicki’s mouth, but there’s an interaction where she tries to put contours around what it means to create community guidelines and rules. What she’s talking about, at least as I hear it, is due process. That word is tossed around a lot but it has meaning: namely, that we have a set of rules created before anything happens, process we use to enforce those rules, and we use that process for everybody.

Due process is important because it creates a sense of fair play and justice. This is more important when we’re talking about laws dealing with peoples’ lives and liberties, but it can be just as meaningful in cultivating a sense of community – a community that people buy into and of which they want to be a part.

Businesses that have poor communities often have poor user experiences, so your service better be exclusive or so exceptional that people eat the cost of having a bad experience with others. Tech companies should take note, in my opinion, that people are being offered more alternatives and developing higher expectations in this area, so I wouldn’t hang my hat on riding this out. Get community moderators, folks. /end self-promotion

In law – the “codifying” to which Wojcicki refers – due process can be boiled down to two core components:

  1. Notice – have you told people how they’re supposed to behave before you enforce that standard of behavior?
  2. Hearing – is there an opportunity to make their case to whomever is enforcing that standard of behavior?

When talk about how to enforce a law, we apply the code to a specific set of facts. Let’s make our use case Logan Paul, who I believe is, at best, an irresponsible and juvenile opportunist who needs to grow up and, at worst, an utterly insensitive knuckledragger with no understanding of common decency towards other people.

Nevertheless, he was a user of YouTube and presumably agreed to abide by their Terms of Service (TOS) which outlined an assortment of rules to which he is bound, including a three-strikes rule, which is exactly what it sounds like. Swisher asks (paraphrase): why don’t you just get rid of him? You make the rules, change the rules.

Agreed, but that doesn’t solve our Logan Paul problem, at least not right away. Paul was bound to a set of rules that, in my opinion, exposed a gap in behavioral expectations for the YouTube platform. It behooves YouTube to change the rules, capture this behavior and close the gap.

What it doesn’t mean is that it’s a good idea to retroactively apply this rule. Remember “notice”: you want to tell people before they act what the expectation is. Removing content that is abhorrent without a codified rationale undermines this principle. There are always going to be exceptions such as an imminent and credible threat to a person’s life, or something so grossly vulgar that the better risk calculation is to take it down and eat the cost of dealing with the aftermath. But, these are exceptions, and we don’t make rules based on exceptional behavior. We make rules based on things that are commonplace and easily understood such that most people find it possible to comply with them.

Additionally, we don’t create rules to target a specific person. It would be dubious to create a rule that seems neutral but, in application, only results in the removal of Logan Paul. Sure, it’s Youtube’s prerogative to remove whomever they want, but I’m coming from a place that assumes YouTube wants to (1) create a consistent user experience; (2) brand as a media platform that doesn’t pick favorites, and; (3) provide a cogent rationale to its stakeholders and users such that they don’t come off as frivolous or erratic.

I’m losing steam since I need to prepare for a meeting, but my roadmap would essentially boil down to the following:

  • Do a gap assessment on YouTube’s rules as of the date of the Logan Paul suicide forest controversy. He may have engaged in questionable behavior in the past but this is the clear marker of what crossed the line in such a way as to enter the cultural zeitgeist and create national controversy.
  • Once you’ve discovered the gap, ask yourself if a rule would have captured this. Sometimes the behavior is so extraordinary that you could make a rule but it wouldn’t, in practice, police anything because it was such a one off. Other times it’s behavior that defies codification. This doesn’t preclude policing it, but it does probably mean you need to preserve in your ToS a level of discretion for content moderators (which you should have) and training for those moderators to spot red flags, etc.
  • Amend the ToS as needed. Make Logan Paul and others click “agree” to participate on your not-a-media-service platform.
  • Penalize all users for non-compliance, including Logan Paul should he run afoul after the new ToS have been socialized.

I don’t have much to say about hearings here. This is something that is a lot more important in traditional legal situations. From what I hear, you can appeal after three strikes and so forth and this is, frankly, a marketplace. You have options to go elsewhere even if they’re shitty options, YouTube isn’t a basic human necessity to which you have some inherent claim.

I can spell “Wojcicki” off-hand now. Boom.

Can the refugee ban help us mitigate the access to justice problem?


Obviously the above picture is not mine. Kamala Khan is Ms. Marvel and belongs to, you guessed it, Marvel. I thought it looked cool and was appropriately in tone and representation.

Spend enough time in law school or at the bar (the boring bar) and you’ll hear about the “access to justice” problem. The access to justice problem, at its most simplest, is that te legal profession cannot sufficiently meet the needs of those who require legal services, and that this disproportionately impacts the poor and indigent. More and more, it is also affecting the working class or lower middle class, who both cannot afford even more middling legal services but also make too much money to qualify for legal aid services.

The salient point is that supply cannot meet demand. There are a lot of reasons for this and we can dive down that rabbit hole another day but, suffice to say, it’s a problem.

Solving problems, in my experience, and I would say in most folks’ experiences, happens incrementally. You make something 3/5/10% better and the eventual outcome is a larger accrued benefit over time.

Likewise, working for a metrics-centric company that values scalable, “machine learning” solutions has changed the way I approach a problem. I’m a lot less interested in the romantic sweeping proposal and a lot more interested in finding gaps that are overlooked when we focus on the main problem. Often, smaller increases in the process can better the quality of life for those part of it and bolster our ability to isolate what the “real” problem is. Instead of getting lost in a sea of possible problems and, consequently, possible solution, we focus on the immediate hurdles we can overcome and then dramatically increase our ability to solve the problem at large by process of isolation and/or elimination.

This isn’t how I’ve seen the legal community solve its problems. For one, it’s averse to change overall. It’s traditional and, by its nature, quite married to precedent, and not just in case law. Moreover, which I’ll touch on later, every solution to a problem must must be at peace with the overarching ethical rules to which lawyers are beholden. Another hurdle to this is that such guidelines are set by state bars, meaning what is ethical in one state might not be in another. Creative solutions to non-substantive problems are thus not only tacitly discouraged by culture but also, as a trade-off to for ethical lawyering, genuine rules.

The Refugee Ban as a Pilot for Better Legal Services

This past weekend, President Trump signed an executive order (“EO”) that “suspended entry of all refugees to the United States for 120 days, barred Syrian refugees indefinitely, and blocked entry into the United States for 90 days for citizens of seven predominantly Muslim countries: Iran, Iraq, Libya, Somalia, Sudan, Syria and Yemen.”

Lawyers quickly camped out at airports. I was not among them. Regretfully, I am licensed in NY and currently unsure of my ability to practice law – even at a federal level – under these circumstances. I’m attempting to clarify it, but nevertheless my participation was limited to Facebook posts and a local rally.

I do have friends who participated. Many have signed up for an Airport Triage. This and similar initiatives seek to compile a list of attorneys who are willing to represent anyone detained or otherwise affected by the EO. It also lists the times they are able to do so, how they can be contacted, and additional information such as whether you can file a habeas corpus petition and what areas of law you typically practice.

Essentially, it’s an intake form. A small, rather thankless but necessary description of who your potential client is, what they need, and how you two can communicate and get together.

It got me wondering about what sort of phone applications one could create. As I mentioned above, this is a very small thing — the sort of thing we really don’t think about when it comes to access to justice. But one hurdle of access to justice is that the people who need lawyers often won’t ask or don’t know where to go. Another is that there are a lot of attorneys who simply cannot afford to do pro bono.

But what about those people who want to do pro bono work and mean to do pro bono work but it gets lost in the sea of other obligations? What if we could lower the barrier to entry for requesting legal aid while providing the sort of information that comes as a push notification and a very short list of facts that, like the refugee ban, galvanizes an attorney

This issue is sexy to us this weekend, but even after the stay and as this is litigated, people will be affected by policies the Trump administration settles on. They are unlikely to change in intent even if they change in process. Programs that keep this and other issues in the minds of attorneys, as well as continue aligning those needs with attorney skills, are worth exploring.

Obviously “I need help with my Earned Income Tax Credit and my husband/wife is a crazy bitch” is not as enticing as protecting the due process rights of a suffering class of people. But the basic thrust is that (1) there are people out there who need lawyers and have engaging/sympathetic/whatever cases, and; (2) there are also a lot of lawyers who do, in fact, want to donate their skills but, like all of us, put it on the backburner until something provides an obvious motivator.

Basically, legal aid societies typically function as the broker for volunteer services. My proposal here, not having considered ethics yet, is to remove the middle man by creating a functional intake app that allows lawyers to screen possible volunteer cases. It’s also a format that is easily accessible to potential clients and is, perhaps, not as daunting as walking into a law firm.

I’m still hashing this out in my head, and I don’t want this post to be too long, so here is a quick list of other off-shoots to this:

  • I’m focusing on a sort of brokerage because apps that focus on being a substantive resource tread too closely to providing legal advice. Providing legal advice can create client relationships one doesn’t want, and depending on who makes the app and populates its information, could run afoul of lawful-practice-of-law, well, laws.
  • Another potential is to use it as a means for attorneys to seek clients who want to operate outside the bounds of a traditional firm. These are probably clients who occupy that working poor/lower middle class range. I’m not sure what the ethical implications are since there are rules about billable hours, but it could function as a means to broker a billable hour they can afford. I’m sure that such a thing would have to be blessed by a state’s ethics bar before this could be done. We don’t want non-lawyers to overshoot how much they should pay only to have a more savvy lawyer take advantage, for example.
  • The unspoken hurdle, particularly in my last point, is cost. Not just cost in development, but cost of offering services. A major pain point in the legal community is that (1) it’s expensive to become an attorney, such that; (2) it’s often cost-prohibitive to work for less than the kind of billable hour that precludes representing kinda-poor people.
  • In general, I’m interested in seeing what other organic solutions to client/attorney hurdles come out of this event. If there’s a silver lining, it might be that necessity begets inventions or otherwise jerry rigged systems that reveal small solutions to larger problems.
  • Of course, my bias is that of a lawyer. We can also consider that refugees and immigrants, as two distinct classes, have some overlapping needs that must be met. Such an app could be used to find people – attorneys or otherwise – who speak their language or are some other kind of advocate.

As someone who is not a developer, that is a major gap in my thinking, but I’m certainly very interested in what other lessons we can learn here – hopefully something more positive, long-term and scalable – to improve access to justice and client experiences. I say this as someone who is, as another disclaimer, no longer has to work with clients so that is yet another blind spot of mine.