How to Build the Good and Responsible Thing
A presentation at Front Conference in in Zürich, Switzerland by Per Axbom
HOW TO BUILD THE GOOD AND RESPONSIBLE THING PER AXBOM • DIGITAL ETHICIST AND COMMUNICATION THEORIST • AXBOM.COM
s r e t e m o l i 6,400 k 7,200 kil ometers
WE NEED TO UNDERSTAND HOW OUR MAPS AND TOOLS INFLUENCE OUR THINKING AND REASONING AND PREJUDICE
Road space taken up by 69 people on a bus on bicycles in cars
Road space taken up by 69 people in fossil-fuel cars in electric cars in self-driving cars
Discover Define Develop Deliver
NORM / “MOST PEOPLE” Discover Define Develop Deliver
NORM / “MOST PEOPLE” Discover Define Develop Deliver
EDGE-CASE NORM / “MOST PEOPLE” EDGE-CASE Discover Define Develop Deliver
MOST PROJECTS KEEP BUILDING FOR THE AVERAGE, NON-EXISTENT, PERSON
We can not keep using the same maps and expect negative impact to suddenly disappear.
The Elements of Digital Ethics Date • version March 1, 2021 • 1.0 Machine Organisation Lack of ethical ownership Digital obfuscation Unaccountability Monoculture Ethicswashing Attention economy Surveillance capitalism Viral reproduction Permanent impact Regulation defects Geographic resistance Supervision Rampant harassment Voice suppression Abuse enablement Exclusion Education defects Ecological neglect Unequal access Environment Society Rule of Quantity Naive recklessness CC-BY-NC-SA Per Axbom • @axbom Technosolutionism Psychoengineering Supply chain neglect Worker endangerment Invisible decisionmaking Algorithmic injustice Digital gaslighting Power concentration Latest + Hi-Res axbom.com/elements Private public spaces online Behavior suppression Human Biometric abuse Privacy devaluation AXBOM.COM/ELEMENTS
HUMAN BEHAVIOR SUPPRESSION Using design to influence people’s behavior in a certain direction, making it harder for them to act in their own interest. The purpose is to have people act in a way that benefits the organisation regardless of a persons’s own goals. Related: Digital gaslighting - making people think any harm is their own fault. https://axbom.com/confirmshaming/
HUMAN EXCLUSION Harm as a result of neglect, unwillingness or inability to consider how our creations affect people’s wellbeing, exclude them from participation or in other ways mislead.
ORGANISATION NAIVE RECKLESSNESS Harm as a result of sensitive information being managed in a way that helps a third party use it for purposes other than what the owners of the information have consented to, or realise they have consented to.
SOCIETY RAMPANT HARASSMENT Harm that happens as a selection of participants in a service harm others and the people responsible for the platform choose not to see or manage the conflict.
ENVIRONMENT SUPPLY CHAIN NEGLECT AND ECOLOGICAL NEGLECT Harm that takes place when organisations do not take responsibility for human welllbeing in the extended ecosystem required to develop or use a service or product.
MACHINE ALGORITHMIC INJUSTICE Harm happening as a result of automated decision systems are developed from a position of power and based on information that is biased or simply incorrect. Since algorithms work faster than humans, algorithmic injustice leads to more efficient execution of harmful decisions.
SOCIETY RULE OF QUANTITY This omnipresent state of mind implies that if something can not be measured it does not carry worth. And the assumption that if something can be measured, it is a clear indicator of worth.
GETTING HURT “BY COMPUTERS” IS NOT NEW
PEOPLE ARE BEING HURT BY COMPUTERS.
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN.
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN PEOPLE.
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN PEOPLE. Invisibly Visibly
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN PEOPLE. Invisibly Visibly
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN PEOPLE. AI.
PEOPLE ARE BEING HURT BY COMPUTERS DESIGN PEOPLE. AI. . e l p o e p l l i t s s ’ t i , h Na
THE WEAKNESSES OF HUMAN REASONING WHY WE KEEP MAKING MODELS AND PROCESSES THAT THEN BITE US IN THE ASS
SYSTEM 2 5% 95% SYSTEM 1 RATIONAL THINKING INTUITION AND INSTINCT Takes effort Slow Logical Lazy Indecisive Unconscious Associative Fast Auto-pilot Popularised in Thinking Fast and Slow by Daniel Kahneman
A HAMBURGER AND A SODA TOGETHER COST 22 SWISS FRANCS. THE HAMBURGER COSTS 20 FRANCS MORE THAN THE SODA. WHAT DOES THE SODA COST?
A HAMBURGER AND A SODA TOGETHER COST 22 SWISS FRANCS. 21 + 1 = 22 THE SODA COSTS 1 SWISS FRANC. THE HAMBURGER COSTS 20 FRANCS MORE THAN THAT (21).
Recognise your excuses for employing SYSTEM 1 IT LOOKS FINE IT IS GOOD ENOUGH FOR NOW IT IS NORMALLY OK, THERE IS NO NEED TO CHECK IT IS NOT MY/OUR RESPONSIBILITY WE ALWAYS DO IT THIS WAY THIS WAY IS QUICKER YOU DECIDE…
<INPUT TYPE=”RESET” VALUE=“CLEAR”> SUBMIT CLEAR
MODELS CAN BE USEFUL. BUT WE ALSO MUST ACKNOWLEDGE THAT THEY ARE ALL WRONG.
DESIGN INTRODUCES UNFAMILIARITY INTO AN EXISTING ECOSYSTEM. WE DESIGN TO MAKE THE SYSTEM WORK DIFFERENTLY THAN IT DID BEFORE.
Cancelled events Other infections/ diseases Mental health distress Domestic violence The systemic complexity of managing a pandemic Drug abuse version PA03 Financial distress Quarantine A Physical distancing If A increases then B increases Lockdown Vaccine) Virus spread Social unrest Health system overload Medical staff falling ill Number of infections False reporting Ignoring health advice B If A increases then B decreases Crime Handwashing Close gatherings A Lay-offs Market closures Population immunity B Number of deaths Examples of the factors at play when looking at how to mitigate spread of disease, and how they may affect one another. The idea is that each factor must be evaluated regularly to support reflective reasoning when making plans of action. Note! This is an example, not a complete picture. Racist attacks System map by Per Axbom April, 2020
NEGATING THE ORIGINAL PURPOSE Platforms designed to bring people closer together are creating more hate and division. Platforms designed to make people safer are putting people in harms way. Platforms designed to help people live healthier lives are giving people anxiety. Platforms designed for inspiration and fun are hurting mental wellbeing.
They say that if a butterfly flaps its wings in the Amazonian rain forest, it can change the weather half a world away. If a web designer changes a button, it can change wellbeing for millions of people across the world. CLEAR
WE WILL ALWAYS BE IN WORK SITUATIONS WHERE THERE IS RISK OF DECISIONS THAT ULTIMATELY HARM PEOPLE. THIS MEANS WE HAVE TO WORK PROACTIVELY TO MANAGE THE WEAKNESSES OF OUR DECISION-MAKING PROCESSES.
WHEN THERE IS SOMETHING BUILT THAT BENEFITS MANY PEOPLE IT CAN ALSO HARM OTHER PEOPLE AT THE SAME TIME.
THE DATA WE USE HARMS PEOPLE Black CIS-Men Hispanic CIS-Women White Asian
DATA IS RARELY “ANONYMOUS” Black CIS-Men Hispanic CIS-Women White Asian Transgender-men Transgender-women
“My transgender student’s legal name is showing on our online discussion board. How can I keep him from being outed to his classmates?” OUR SOURCE SYSTEMS ARE NOT ALWAYS HELPFUL https://alistapart.com/article/trans-inclusive-design/
SPEAKING OF NAMES PEOPLE HAVE EXACTLY ONE FULL NAME WHICH THEY GO BY. PEOPLE HAVE EXACTLY X NAMES, FOR ANY VALUE OF X. PEOPLE’S NAMES ARE WRITTEN IN ANY SINGLE CHARACTER SET. PEOPLE’S NAMES ARE NOT WRITTEN IN ALL CAPS. I CAN SAFELY ASSUME THAT A DICTIONARY OF BAD WORDS CONTAINS NO PEOPLE’S NAMES IN IT. NO MILLION PEOPLE SHARE THE SAME FULL NAME. PEOPLE HAVE NAMES. https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/
THE NULL FAMILY NAME Radiolab episode https://www.bbc.com/future/article/20160325-the-names-that-break-computer-systems
IF NULL THEN… IF “NULL” THEN…
MAYBE YOU COULD CHANGE YOUR NAME? A very real suggestion from an insurance company
IT IS GOOD ENOUGH FOR NOW
THOSE ARE JUST EDGECASES, NOT THE NORM WE’VE DONE WHAT WE CAN OTHERS ARE ALREADY DOING IT
THERE WAS AN OBVIOUS SPEAKING PARTNER WITH SUPERIOR KNOWLEDGE IN ALL THIS, LET’S HOPE THEY REACH OUT TO US…
MEANWHILE, ON THE INTERNET
AT LEAST ROB CAN FIND HIS KEYS
NEW SERVICES AND APPS PROVIDE A REMOTE CONTROL FOR ABUSE. EXTREMELY DIFFICULT TO POLICE. ONE MAIN ACCOUNT OWNER. CAN CONTINUE LONG AFTER PEOPLE “LEAVE” A RELATIONSHIP.
PASSWORDS DO NOT KEEP THINGS PRIVATE. OWNING A DEVICE DOESN’T KEEP OTHERS FROM USING IT.
AS THE MAKERS OF DIGITAL SERVICES AND PRODUCTS, WE NEED TOOLS AND MAPS TO MANAGE AND MITIGATE OUR OWN WEAKNESSES THESE PROBLEMS ARE SOLVABLE
Obvious potential for negative impact
Obvious potential for negative impact Negative impact only discovered after it happens
Obvious potential for negative impact Negative impact predicted by speculative, systematic reasoning and strategic foresight Negative impact only discovered after it happens
Obvious potential for negative impact Negative impact predicted by speculative, systematic reasoning and strategic foresight Negative impact predicted by involving more people with relevant experience Negative impact only discovered after it happens
IMPACT CARDS Who? ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ Ability divergent Age disfavored Appearance disfavored Crime/distress endurer Ethnic minority Faith disfavored Gender disfavored Illness endurer Non-citizen Racism endurer Social class disfavored Sexuality disfavored What? ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ Environment Esteem Finance Health Privacy Relationships Safety Self-actualization Self-worth Social belonging How? ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ ▢ Dangerous defaults False assumptions False expectations Excluded by design External pressure Lack of choice Lack of knowledge Lack of understanding Sensory overload Time constraints version 0.9 of the cards
STORYMAPPING 4. CLIMAX 1. PERSON 2. GOAL/PROBLEM 5. GAIN 3. ENCOUNTER Design/solution/product
MAPPING THE OTHER STORY 4. CLIMAX/CRISIS 1. PERSON 2. GOAL/PROBLEM 5. IMPACT/GAIN/LOSS 3. ENCOUNTER Design/solution/product
STORYMAPPING 36 yo developer Tries out exposure exercises 4. CLIMAX
- PERSON social anxiety 2. GOAL/PROBLEM Makes a new friend AI therapist app 3. ENCOUNTER Design/solution/product
- GAIN
MAPPING THE OTHER STORY Unloads on the AI therapist Child 4. CLIMAX/CRISIS
- PERSON Never helped Abusive parents 2. GOAL/PROBLEM AI therapist app
- ENCOUNTER Design/solution/product
- IMPACT/GAIN/LOSS
INCLUSIVE PANDA
INCLUSIVE PANDA (non-participants) excluded unwanted participants risk zone included risk-zone negative impact
INCLUSIVE PANDA (non-participants) excluded unwanted participants risk zone included risk-zone negative impact Use to understand effects on other people than the target group.
IMPACT MAPPING Who? people who want to protect their identity How bad/severe is the potential outcome? How much do we contribute through what we build? How likely is it to happen? How disadvantaged are the people affected? abusers/attackers find them and hurt them What?
IMPACT MAPPING Who? people who want to protect their identity abusers/attackers find them and hurt them What? d n a How bad/severe is the st r e d n potential outcome? u n o i s t e a i t s i i v n i a t c g r a How much do we contribute o e r s i u t i r through what we build? lp yo o i r e p h d o n t a e t s c U a p How likely is it to happen? im How disadvantaged are the people affected?
Tarot cards of Tech http://tarotcardsoftech.artefactgroup.com
FRICTION Force considered decisions
FRICTION Prevent misuse / abuse
FRICTION You will find the password by reading the welcome information carefully. g in g g lo r fo d r o w s s a p e th d n fi In this film you will r u o Y . s r te c a r a h c ll a m s 5 , in to the booking system . il a -m e n o ti a m r fi n o c e th in username is Code of conduct explained whilst giving out one letter of the password every thirty seconds.
PRE-SCRIPTING We need to do this. ALTERNATIVE PATHS ADDICTIVE CYCLES THE OTHER STORY Hang on, I’m concerned about the impact of this decision. Maybe you have the answer. SHORT TERM / LONG TERM COST CONSIDERATION REPUTATION AND TRUST
Everywhere Can often be anticipated if you know where/when they tend to show up ETHICAL DILEMMAS ARE LIKE BIRDS Varied Guides and tools can help Easier to spot in groups Concentrated in particular environments Noticed by people who are in the habit of looking for them https://www.scu.edu/ethics/focus-areas/technology-ethics/ Difficult to spot
https://www.scu.edu/ethics/focus-areas/technology-ethics/ GET INTO THE HABIT OF LOOKING FOR THEM!
Who is in the room and making the decisions? Am I sure the design pattern that worked over there also works here? Do we really need to ask for this data? What faulty assumptions are we making? Instead of making decisions FOR people, how could we actually build something differently to support people in exercising their autonomy and self-determination, to activate system 2 thinking? Am I inside an environment that supports ethical reasoning? Who do we need to talk to? https://www.scu.edu/ethics/focus-areas/technology-ethics/ GET INTO THE HABIT OF LOOKING FOR THEM!
ETHICAL DESIGN IS NOT ABOUT STOPPING TECHNICAL INNOVATION. IT’S ABOUT INFLUENCING ITS DIRECTION. (WITH BETTER MAPS AND INDICATORS) harmful to many harmful to some promotes wellbeing harmful to some harmful to many
ETHICAL DESIGN IS NOT ABOUT STOPPING TECHNICAL INNOVATION. IT’S ABOUT INFLUENCING ITS DIRECTION. (WITH BETTER MAPS AND INDICATORS) harmful to many harmful to some promotes wellbeing harmful to some harmful to many
PREDICTION:
ALL COMPANIES WILL BE OUTPERFORMED BY A MORE ETHICAL VERSION CATERING TO THE EXACT SAME NEEDS.
ALL COMPANIES WILL BE OUTPERFORMED BY A MORE ETHICAL VERSION CATERING TO THE EXACT SAME NEEDS. ARE YOU DOING YOUR BEST WORK?
PER AXBOM axbom.com axbom.com/frontzurich d n a s k n i l S lides, s e c n e r ref e d n i k e b o t r e b m e Rem f l e s r u o y to !
A practical and concrete introduction to the space of digital ethics and responsible innovation for everyone who designs and builds digital products. You will learn
- why ethics is important and valuable,
- how to predict and manage harms to wellbeing, and
- the tricks that give power, autonomy and justice back to the people we wish to help
You will also begin to understand how you can be a voice of reason and gain traction for ethical thinking within your organisation.
Per promises lots of real-world examples to get to the essence of an ethical mindset.
