Would you tell Facebook you’re happy to see all the bared flesh it can show you? And that the more gratuitous violence it pumps into your News Feed the better?

Obtaining answers to where a person’s ‘line’ on viewing what can be controversial types of content lies is now on Facebook’s product roadmap — explicitly stated by CEO Mark Zuckerberg in a lenthy blog post last week, not-so-humbly entitled ‘Building a global community‘.

Make no mistake, this is a huge shift from the one-size fits all ‘community standards’ Facebook has peddled for years — crashing into controversies of its own when, for example, it disappeared an iconic Vietnam war photograph of a naked child fleeing a napalm attack.

In last week’s wordy essay — in which Zuckerberg generally tries to promote the grandiose notion that Facebook’s future role is to be the glue holding the fabric of global society together, even as he fails to flag the obvious paradox: that technology which helps amplify misinformation and prejudice might not be so great for social cohesion after all — the Facebook CEO sketches out an impending change to community standards that will see the site actively ask users to set a ‘personal tolerance threshold’ for viewing various types of less-than-vanilla content.

On this Zuckerberg writes:

The idea is to give everyone in the community options for how they would like to set the content policy for themselves. Where is your line on nudity? On violence? On graphic content? On profanity? What you decide will be your personal settings. We will periodically ask you these questions to increase participation and so you don’t need to dig around to find them. For those who don’t make a decision, the default will be whatever the majority of people in your region selected, like a referendum. Of course you will always be free to update your personal settings anytime.

With a broader range of controls, content will only be taken down if it is more objectionable than the most permissive options allow. Within that range, content should simply not be shown to anyone whose personal controls suggest they would not want to see it, or at least they should see a warning first. Although we will still block content based on standards and local laws, our hope is that this system of personal controls and democratic referenda should minimize restrictions on what we can share.

A following paragraph caveats that Facebook’s in-house AI does not currently have the ability to automatically identify every type of (potentially) problematic content. Though the engineer in Zuck is apparently keeping the flame of possibility alive — by declining to state the obvious: that understanding the entire spectrum of possible human controversies would require a truly super-intelligent AI.

(Meanwhile, Facebook’s in-house algorithms have shown themselves to be hopeless at being able to correctly ID some pretty bald-faced fakery. And he’s leaning on third party fact-checking organizations — who do employ actual humans to separate truth and lies — to help fight the spread of Fake News on the platform, so set your expectations accordingly… )

“It’s worth noting that major advances in AI are required to understand text, photos and videos to judge whether they contain hate speech, graphic violence, sexually explicit content, and more. At our current pace of research, we hope to begin handling some of these cases in 2017, but others will not be possible for many years,” is how Zuck frames Facebook’s challenge here.

The problem is this — and indeed much else in the ~5,000-word post — is mostly misdirection.

The issue is not whether Facebook will be able to do what he suggests is its ultimate AI-powered goal (i.e. scan all user-shared content for problems; categorize everything accurately across a range of measures; and then dish up exactly the stuff each user wants to see in order to keep them fully engaged on Facebook, and save Facebook from any more content removal controversies) — rather the point is Facebook is going to be asking users to explicitly give it even more personal data.

Data that is necessarily highly sensitive in nature — being as the community governance issue he’s flagging here relates to controversial content. Nudity, violence, profanity, hate speech, and so on.

Yet Facebook remains an advertising business. It profiles all its users, and even tracks non-users‘ web browsing habits, continually harvesting digital usage signals to feed its ad targeting algorithms. So the obvious question is whether or not any additional data Facebook gathers from users via a ‘content threshold setting’ will become another input for fleshing out its user profiles for helping it target ads.

We asked Facebook whether it intends to use data provided by users responding to content settings-related questions in future for ad targeting purposes but the company declined to comment further on Zuckerberg’s post.

You might also wonder whether, given the scale of Facebook’s tracking systems and machine learning algorithms, couldn’t it essentially infer individuals’ likely tolerance for controversial content? Why does it need to ask at all?

And isn’t it also odd that Zuckerberg didn’t suggest an engineering solution for managing controversial content, given, for example, he’s been so intent on pursuing an engineering solution to the problem of Fake News. Why doesn’t he talk about how AI might also rise to the complex challenge of figuring out personal content tastes without offending people?

“To some extent they probably can already make a very educated, very good guess at [the types of content people are okay seeing],” argues Eerke Boiten, senior lecturer in computer science at the University of Kent. “But… telling Facebook explicitly what your line in the sand is on different categories of content is in itself giving Facebook a whole lot of quite high level information that they can use for profiling again.

“Not only could they derive that information from what they already have but it would also help them to fine-tune the information they already have. It works in two directions. It reinforces the profiling, and could be deduced from profiling in the first place.”

“It’s checking their inferred data is accurate,” agrees Paul Bernal, law lecturer at the University of East Anglia. “It’s almost testing their algorithms. ‘We reckon this about you, this is what you say — and this is why we’ve got it wrong’. It can actually, effectively be improving their ability to determine information on people.”

Bernal also makes the point that there could be a difference, in data protection law terms, if Facebook users are directly handing over personal information about content tolerances to Facebook (i.e. when it asks them to tell it) vs such personal information being inferred by Facebook’s indirect tracking of their usage of its platform.

“In data protection terms there is at least some question if they derive information — for example sexuality from our shopping habits — whether that brings into play all of the sensitive personal data rules. If we’ve given it consensually then it’s clearer that they have permission. So again they may be trying to head off issues,” he suggests. “I do see this as being another data-grab, and I do see this as being another way of enriching the value of their own data and testing their algorithms.”

“I’m not on Facebook and this makes it even clearer to me why I’m not on Facebook because it seems to me in particular this is increasing risks and increasing our vulnerability at a time when we should be doing exactly the opposite,” adds Bernal.

Facebook users are able to request to see some of the personal data Facebook holds on them. But, as Boiten points out, this list is by no means complete. “What they give you back is not the full information they have on you,” he tells TechCrunch. “Because some of the tracking they are doing is really more sophisticated than that. I am absolutely, 100 per cent certain that they are hiding stuff in there. They don’t give you the full information even if you ask for it.

“A very simple example of that is that they memorize your search history within Facebook. Even if you delete your Facebook search history it still autocompletes on the basis of your past searches. So I have no doubt whatsoever that Facebook knows more than they are letting on… There remains a complete lack of transparency.”

So it at least seems fair that Facebook could take a shot at inferring users’ content thresholds, based on the mass of personal data it holds on individuals. (One anecdotal example: I recall once seeing a notification float into my News Feed that a Facebook friend had liked a page called “Rough sex”, which would seem to be just the sort of relevant preference signal Facebook could use to auto-determine the types of content thresholds Zuckerberg is talking about, at least for users who have shared enough such signals with it.)

But of course if it did that Facebook would be diving headfirst into some very controversial territory. And underlining exactly how much it knows about you, dear user — and that might comes across as especially creepy when paired with a News Feed that’s injecting graphic content into your eyeballs because it thinks that’s what you want to see.

“Given the level at which they’re profiling we shouldn’t tell them anymore,” says Boiten, when asked whether people should feel okay feeding Facebook info about their ‘line in the sand’ — pointing to another controversy that arose last year when it emerged Facebook’s ad capabilities could be used to actively exclude or include people with specific ethnic affinities (aka ‘racial profiling’).

“If they make the advances in understanding of natural language content — the AI slant that Zuckerberg’s [blog post] promises, probably unrealistically, but nevertheless — if they get that sort of advantage then blimey they’re going to know an awful lot more than they already do,” he adds.

“You can bet that they’re going to be profiling people based on their standards settings in this way,” adds Bernal. “That’s how it works, and then they aggregate it and they’ll be using it — I bet — to target their advertising and so on. It is more total information management. The more they can get, and the more granular those personal controls get the more information they’re picking up.

“And I do think it’s disingenuous in that Zuckerberg’s post is not mentioning any of this.”

While it’s not yet clear exactly how (or when) these content settings will be implemented, the structure sketched out by Zuckerberg already looks pretty problematic — given that Facebook users who do not want to share any additional sensitive signals with the ad-targeting giant will be forced to tolerate their peers’ predilections.

Which immediately puts pressure on users to confess their content likes/dislikes to Facebook in order to avoid this ‘hell is other people’s tastes’ bind — i.e. in order to not be subject to the preferences of a local median. And to avoid being tainted by association of the types of content showing up (or not showing up) in their News Feed. After all, the Facebook News Feed is inherently individual — so there’s a risk of the character of the content in a user’s Feed being assumed to be a reflection of their personal tastes.

So by not telling Facebook anything about your content thresholds you’re put into a default corner of telling Facebook you’re okay with whatever the regional average is okay with, content wise. And that may be the opposite of okay for you.

“I think there’s another little trap here that they’ve done before,” continues Bernal. “When you make controls granular it looks as if you’re giving people control — but actually people generally get bored and don’t bother changing anything. So you can say you’ve given people control, and now it’s all much better — but in general they don’t use it. The few people who do are the few people who would understand it and get round it anyway. It will be very interesting to see what extent people actually use it.”

Such a majority rule system could also be at risk of being gamed by — let’s say — mischievous 4Channers banding together and working to get graphic boundaries opened up in a region where more conservative standards are the norm.

“I can see people gaming this kind of system — in the way that all kinds of online polls and referenda are gamed, somebody will work out the way to get the systems set the way they want… There are all kinds of possibilities,” argues Bernal. “There’s also a danger of ‘community leaders’ taking some degree of control; recommending people particular settings. I’m wary of Zuckerberg ending up doing this so you have standards for particular kind of people, so you ‘chose’ the standards that someone else has effectively chosen for you.”

A lot will depend on the implementation of the content controls, certainly, but when you look at how easily, for example, Facebook’s trending news section — not to mention its News Feed in general — has been shown to be vulnerable to manipulation (by purveyors of clickbait, Fake News etc) it suggests there could well be risks of content settings being turned on their head, and ending up causing more offenses than they were trying to prevent.

Another point Bernal makes is that shifting some of the responsibility for the types of content being shown onto users implicitly shifts some of the blame away from Facebook when controversies inexorably arise. So, basically: see something you don’t like in your News Feed in future? Well, that’s YOUR fault now! Either you didn’t set your Facebook content settings correctly. Or you didn’t set any at all… Tsk!

In other words, Facebook gets to deflect objections to the type of content its algorithms are shunting into the News Feeds of users all over the world as a ‘settings configuration’ issue — sidestepping having to address the more systemic and fundamental flaw embedded into the design of the Facebook product: aka the filter bubble issue.

Facebook has long been accused of encouraging a narrowing of personal perspective via its user-engagement focused content hierarchies. And Zuckerberg’s blog post has a fair amount of fuzzy thinking on filter bubbles, as you might expect from the chief of an engagement-algorithm-driven content distribution machine. But — for all his talk of “building global community” — he offers no clear fix for how Facebook can help break users out of the AI-enabled, navel-gazing circles its business model creates.

Yet a very simple fix for this does exist — which would be to show people a chronological News Feed of friends’ posts as the default vs the current default being the algorithmically powered one. Facebook users can manually switch to a chronological feed but the option is tricky to find, and clearly actively discouraged as the choice gets reset back to the AI Feed either per session or very soon after. In short the choice barely exists.

The root problem here of course is that Facebook’s business benefits massively from algorithmically engaged users. So there’s zero chance Zuck is going to announce it’s abandoning such a lucrative and (thus far) scalable default. So his solitary claim in the essay to “worry” about fake news and filter bubbles rings very hollow indeed.

Indeed, there is also a risk that giving users controls over controversial content could exacerbate the filter bubble effect further. Because a user who can effectively dial down all controversy to zero is probably not going to be encountering news about conflict in Syria, say. It’s going to be a lot easier for them to live inside a padded Facebook stream populated with cute photos of babies and kittens. News? What news? Awwww, how purdy!

And while that might make a pleasing experience for individuals who wants to disengage from wider global realities, it’s reductive for society as a whole if lots of people start retreating into rose-tinted filter bubbles. (Dialing up hateful content, should that also be possible via the future Facebook content filters, would also obviously likely have a deleterious and divisive societal impact).

The point is, giving people easy opt outs for types of content that might push them outside their comfort zone and force them to confront unfamiliar ideas or encounter a different or difficult perspective just offers a self-enabled filter bubble (alongside the algorithmic filter Facebook users get pushed inside when inside Facebook, thanks to its default setting).

This issue is of rising important given how many users Facebook has, and how the massively dominant platform has been shown to be increasingly cannibalizing traditional news media; becoming a place people go to get news generally, not just to learn what their friends are up to.

And remember, all this stuff is being discussed in a post where Zuckerberg is seeking to position Facebook as the platform to glue the world together in a “global community” and at a fractious moment in history. Which would imply giving users the ability to access perspectives far-flung from their own, rather that helping people retreat into reductive digital comfort zones. A multitude of disconnected filter bubbles certainly does not have the ring of ‘global community’ to me.

Another glaring omission in Zuckerberg’s writing is the risk of Facebook’s cache of highly personal (and likely increasingly sensitive) data being misused by overreaching governments seeking to clamp down on particular groups within society.

It’s especially strange for a US CEO to stay silent on this at this point in time, given how social media searches by US customs agents have ramped up following President Trump’s Executive Order on immigration last month. There have also been suggestions that foreigners wanting to enter the US could be forced to hand over their social media passwords to US border agents in future. All of which has very clear and very alarming implications for Facebook users and their Facebook data.

Yet the threat posed to Facebook users by government agencies appropriating accounts to enable highly intrusive, warrantless searches — and presumably go on phishing expeditions for incriminating content, perhaps, in future, as a matter of course for all foreigners traveling to the US — apparently does not merit public consideration by Facebook’s CEO.

Instead, Zuckerberg is calling for more user data, and for increased use of Facebook.

While clearly such calls are driven by the commercial imperatives of his business, the essay is couched as a humanitarian manifesto. So those calls seems either willfully ignorant or recklessly disingenuous.

I’ll leave the last word to Bernal: “The idea that we concentrate all our stuff in one place — both in one online place (i.e. Facebook) and one physical place (i.e. our smartphones), puts us at greater risk when we have governments who are likely to take advantage of those risks. And are actually looking at doing things that will be putting us under pressure. So I think we need to be looking at diversifying, rather than looking at one particular route in.

“Anyone who’s got any sense is not going to be doing anything that’s even slightly risky on Facebook,” he adds. “And should be looking for alternatives. Because while the border guards may know about Facebook and Twitter they’re not going to know about the more obscure systems, and they’re not going to be able to get access to them. So now is actually the time for us to be saying let’s do less Facebook, not more Facebook.”